Commit Graph

264 Commits

Author SHA1 Message Date
Jan Tattermusch b436d0dfb7
Improve emulated linux aarch64 tests, without protoc artifact build
Based on my testing, the tests are currently flaky, so it would be better to first merge it and start running it continuously to asses the situation.
2021-04-26 10:54:28 -07:00
Lidi Zheng 134e9cbc42
buildscripts: enable CSDS test 2021-04-21 20:48:34 -04:00
Sergii Tkachenko 278a336d1f buildscript: xds-k8s increase build timeout 2021-04-08 22:05:29 -04:00
Sergii Tkachenko c113ba1030 buildscript: add xds-k8s cluster endpoint override
Missed this in 1b86618ce9
2021-04-08 21:14:08 -04:00
Sergii Tkachenko 1b86618ce9 buildscript: use different xds-k8s cluster
In preparation to the Public Preview.
2021-04-08 19:00:34 -04:00
ZHANG Dapeng e4ccf0eeac
buildscript: enable fault injection interop test 2021-03-17 17:41:58 -07:00
Doug Fawley 3752b9e365
add timeout test case to xds interop tests (#7888) 2021-02-12 14:25:20 -08:00
Eric Gribkoff 0ffd942fd2
buildscripts: increase xds job timeouts (#7831) 2021-01-25 04:33:01 -08:00
ZHANG Dapeng 150f994315
buildscript: fix execution path of xds.sh in xds_v3.sh 2021-01-20 09:21:36 -08:00
ZHANG Dapeng c1f9658a72
buildscripts: fix buildscript unbound variable error 2021-01-19 18:33:55 -08:00
ZHANG Dapeng 56fc8b8626
buildscripts: add xds_v3 buildscripts 2021-01-19 15:01:21 -08:00
Jan Tattermusch 76c5a651a1
Add a simple way to run grpc-java test inside emulated ARM64 docker container 2021-01-19 09:05:22 -08:00
Sergii Tkachenko e5f57adb7f
buildscripts: xDS Kubernetes Interop tests buildscript 2020-12-23 09:52:01 -05:00
Chengyuan Zhang 1e49ac665a
buildscript: ensure dependency convergence (#7745)
Run grpc-all:dependency task in Kokoro. For some unknown reason, building grpc-all doesn't reliably catch dependency conflicts.
2020-12-22 09:09:11 -08:00
Eric Anderson 8ce6355e89 buildscripts: Combine android builds together
Previously the android projects were separate from the main build and
each other. For quite a while now they have been integrated in the main
project. There's no longer any need to build each separately.
2020-12-07 15:01:51 -08:00
Chengyuan Zhang 71a3c55177
buildscripts: add missing CI coverage for examples (#7708)
Adds CI coverage for building example/android/strictmode and examples/example-jwt-auth. Also cleans up existing Gradle and Maven build command in the CIs.
2020-12-07 14:43:32 -08:00
Eric Anderson f13471d68d buildscripts: Check for out-of-date codegen after android builds
Issues like that fixed in 05048cf3 should have failed the CI, like it
does for the rest of the build (on both Travis and linux_artifacts).
2020-12-07 11:56:06 -08:00
Eric Anderson ee9286a0f2 buildscripts: Fix grpc-java-artifacts indentation
Re-sort packages, now that the tabs aren't skewing the sorting.

I'm quite confident I had fixed this already, but I had multiple copies
of this file on multiple machines and must have fixed the wrong copy.
2020-12-04 16:42:26 -08:00
Eric Anderson 486d82a7e1 Build Linux artifacts using CentOS 7
CentOS 6 is dead and no longer has update servers. CentOS 7 is older
than Debian 9 (oldstable), so binaries hopefully work on both. More
testing is necessary, but everything's broken now, so this is better
than nothing.

We stop using protoc-artifacts because now the container is
straight-forward enough that we can just use our own. Previously the
"devtoolset" stuff made us want to share the container.
2020-12-03 22:56:31 -08:00
Sergii Tkachenko fd59f1de0c buildscripts: Add xDS GKE interop tests buildscript placeholder 2020-11-25 16:24:07 -05:00
Chengyuan Zhang c850840342
buildscripts: enable xDS circuit breaking test (#7615) 2020-11-16 19:27:57 -08:00
Chengyuan Zhang da100e8e49
build, examples, README.md: Update protobuf gradle plugin version to 0.8.13 (#7355)
Updated protobuf gradle plugin version to 0.8.13. Fixed Android Kokoro's memory issue by forcing to use a new Gradle daemon for building the previous commit.
2020-09-21 10:42:06 -07:00
Eric Anderson 96a5c5df57 buildscripts: Avoid signing sha256 and sha512 checksums
Starting in Gradle 6.0 maven-publish began including sha256 and sha512
checksums, in addition to the previous md5 and sha1 checksums. We don't want
.sha256.asc and .sha512.asc files, as they serve no purpose.
2020-07-31 09:52:36 -05:00
Menghan Li d2182fe197
interop-testing: add path_matching and header_matching (#7254) 2020-07-28 14:07:43 -07:00
Eric Gribkoff 4fbe6bef7d
interop-testing: add flags to xds test client 2020-07-28 12:43:00 -07:00
Eric Anderson 8a792c3367 buildscripts: Fix check for java-example-hostname:latest tagging
The previous 'git ls-remote' was returning the tag-referenced commits via
vF.O.O^{} which was confusing the version check (as v1.30.0 would not match
v1.30.0^{}). Passing --refs filters those ^{} tags.

There was also a missing 'v' in the version check. It was added explicitly to
generate the list but not to check the result.
2020-07-24 09:22:05 -05:00
ZHANG Dapeng 0d6546719a
all: bump protobuf to 3.12.0
Version 3.12.0 is needed for the feature of support for proto3 field presence #7051
2020-05-20 16:30:59 -07:00
Eric Anderson b1ca6e8d27 Automate release process for example-hostname 2020-05-12 14:30:27 -07:00
Eric Gribkoff 95b0550c59
buildscripts: use xds-test-server image (#6889) 2020-04-23 15:59:19 -07:00
Patrice Chalin da855f4164
Delete example-kotlin (#6936) 2020-04-16 13:55:05 -07:00
ZHANG Dapeng 5c31dc6d71
xds: migrating XdsNameResolver to "xds" scheme
Change "xds-experimenal" scheme to "xds" and add a deprecated provider with "xds-experimenal" scheme for keeping current users' tests working.
2020-04-07 11:52:41 -07:00
Eric Gribkoff e2bb44106a
buildscripts: allow per-xds test case artifacts (#6857) 2020-03-26 10:13:05 -07:00
Chengyuan Zhang 95e331a69b
buildscripts: clean up android related scripts (#6843)
Fix incorrect android-interop-testing apk path after flatten the project structure.
2020-03-19 19:09:54 -07:00
Eric Gribkoff 1acde4604f
buildscripts: checkout matching grpc/grpc branch for xds test (#6828) 2020-03-13 15:32:13 -07:00
Eric Gribkoff 99b6d0156e
buildscripts,interop-testing: Increase logging for xDS tests (#6818) 2020-03-11 20:34:26 -07:00
Eric Gribkoff 3b8e36358c
buildscripts: update xds test server arg (#6816) 2020-03-11 13:20:30 -07:00
Chengyuan Zhang 5ba663bcf6
android: add grpc-android into main build (#6793)
Add grpc-android into main build. grpc-android will be built if Gradle option skipAndroid is false. This change also migrates deprecated Robolectric methods to androidx.test methods.
2020-03-04 15:40:39 -08:00
Jihun Cho 0fd4975d4c
rls: add proto and converter (#6743) 2020-02-28 21:41:19 -08:00
Eric Gribkoff 5555ec9a35
buildscripts: fix url of grpc core repo (#6708) 2020-02-13 18:51:08 -08:00
Eric Gribkoff 84b10350eb buildscripts,interop-testing: Add xDS tests on GCP (#6662) 2020-02-09 19:43:11 -08:00
Eric Anderson 2e12b53f2f
examples: Add XDS client example 2020-01-31 11:13:44 -08:00
Eric Anderson e7d7c5bde4
examples: Add hostname example
This is placed in its own directory since it depends on grpc-services.
2020-01-17 15:50:49 -08:00
Eric Anderson 8cf0196548
Add Cronet to main build
Building now requires an Android SDK unless you specify
-PskipAndroid=true
2020-01-09 15:20:41 -08:00
Eric Anderson c606519a5d bazel: Support maven_install
maven_install is strongly superior to previous forms of grabbing dependencies
from Maven as it computes the appropriate versions to use from the full
transitive dependencies of all libraries used by an application. It also has
much less boilerplate and includes dependencies with generated targets.

In the future we will drop the jvm_maven_import_external usages and require
maven_install, at which point we can swap to using the `@maven' repository and
no longer depend on compat_repositories.

Fixes #5359
2019-12-30 12:08:42 -08:00
Liu sheng 470a4ab12d Add support for cross-compiling for aarch64 platform
This change modified the building scripts to support cross compile
grpc-java for aarch64 platform

For grpc/grpc-java#6364
2019-12-06 14:21:34 -08:00
Eric Anderson 54b7847a7f Update protobuf to 3.11.0 and gson to 2.8.6
Updating Protobuf to bring in improvements for Android.

Protobuf pulls in the newer version of gson and we are happy to update
to it.
2019-12-02 16:43:06 -08:00
Elliotte Rusty Harold 079219bbc9 Update to GSON 2.8.5 and protobuf 3.10.0 2019-10-09 07:47:34 -07:00
Chengyuan Zhang e866d3539c
buildscripts: add config for building grpc-cronet artifact (#6134) 2019-09-09 16:14:56 -07:00
ZHANG Dapeng c240f27077
buildscripts: android.sh to clean the build when building HEAD^
A gradle plugin [issue](google/protobuf-gradle-plugin#331) was seen for https://github.com/grpc/grpc-java/pull/6099. This PR try to workaround it.
2019-08-28 12:56:49 -07:00
Carl Mastrangelo 57e7bd394e
all: update to proto 3.9.0 2019-07-26 13:53:05 -07:00
Eric Anderson be819fa3fd
android: Convert to maven-publish
com.github.dcendents:android-maven-gradle-plugin is incompatible with
Gradle 5 and the project hasn't seen any activity in over a year, so it
seems unlikely to get fixed.

We want to use maven-publish anyway, since that's what we use elsewhere.
2019-06-06 14:53:27 -07:00
Eric Anderson 52dff83717 Update Protobuf to 3.7.1
This mainly avoids protoc from 3.7.0 which has a dependency on libatomic. Most
of our systems have libatomic, so it mostly works, but the interop docker
container does not, so building fails. Version 3.7.1 was rebuilt to avoid
needing the libatomic shared library.

This has the added benefit that Bazel is now on the same version as Gradle, as
3.7.1 included fixes for Bazel.
2019-04-05 10:55:14 -07:00
ZHANG Dapeng a17f8ab62c
buildscripts: add android-interop-test build (without runing test) in kokoro android 2019-04-03 14:22:39 -07:00
ZHANG Dapeng 755a22790b
android/android-interop-testing/examples: upgrade android plugin
This resolves #5523

While bumping `com.android.tools.build:gradle:3.1.2` to `3.3.0`, some other plugins/artifacts/maven repo/buildscripts have to be updated:

- gradle (wrapper) need to upgrade to 4.10.x
- protobuf gradle plugin need to bump a version compatible with gradle version.
- need add `google()` and `jcenter()` repos for android (otherwise `com.android.tools.build:aapt2:3.3.0x` and `trove4j` will not be found resp.)
- need to accept license for Android "build-tools;28.0.3" in kokoro env.
2019-04-02 14:52:27 -07:00
Eric Anderson e7a635d882
buildscripts: Update protoc-artifacts to 3.7.0
This was missed from d7e53e871

Also added missing --tmpdir, to avoid extracting protobuf
into the current directory.
2019-03-13 13:43:21 -06:00
Eric Anderson d7e53e871b
Merge pull request #5454 from ejona86/protobuf-3.7.0
Upgrade to Protobuf 3.7.0
2019-03-11 15:39:50 -06:00
Liam Miller-Cushon 952a767b9c Rename source jar outputs to avoid conflicts with java_common.compile
More information: https://github.com/bazelbuild/bazel/issues/5824
2019-03-05 08:44:22 -07:00
Rodrigo Queiro 0959a846c8 bazel: update to new ProtoInfo provider
This is part of #5383. As Bazel 0.22.0 is the only supported version
now, use this in the Kokoro build.
2019-02-28 21:30:48 -07:00
Jihun Cho 68f6ce9ddd
all: fix publish not having checksum (#5398) 2019-02-27 09:39:02 -08:00
Eric Anderson eaca73473c
Upgrade to protobuf 3.6.1
For Bazel, we upgrade to protobuf 3.6.1.2 and javalite HEAD to fix
incompatibilities in newer Bazel releases.

compiler/Dockerfile is unused, so it was removed instead of being updated.

protoc no longer includes codegen for nano, so we remain on the older protoc
any time nano is used.

Protobuf now requires C++11 when compiling, so windows was swapped to
VC 14.
2019-02-07 13:40:53 -08:00
Jihun Cho 1c3432c3fb
all: migrate gradle publish from maven to use maven-publish plugin (#5289) 2019-01-31 17:38:43 -08:00
Eric Gribkoff f973bbc06f
Revert "buildscripts: disable android size status check (#5278)" (#5284)
This reverts commit aaa3a86dd2.
2019-01-25 16:16:41 -08:00
Eric Gribkoff aaa3a86dd2
buildscripts: disable android size status check (#5278) 2019-01-24 12:21:58 -08:00
ZHANG Dapeng 788cdbd511
buildscripts: Show bazel version in CI 2019-01-14 16:54:51 -08:00
Eric Anderson c8817cf284
kokoro: Reduce gradle workers on Mac to reduce flakes
We've had a long run of test timeout failures on Mac with timeouts in
the seconds. Multiple of these haven't seemed like races, but simply
that the Mac machines were too slow. The most recent case for this is a
set of timeouts for
OkHttpTransportTest.earlyServerClose_serverFailure_withClientCancelOnListenerClosed
where it took over a second for a server stream to be created ("Timed
out waiting for server stream").

This commit reduces the number of workers to reduce the overall load,
hoping this lets tests complete within a reasonable amount of time.
2019-01-11 08:56:45 -08:00
Eric Anderson 186409f868
buildscripts: Add script to sync .protos from grpc-proto
Currently only protos in alts, grpclb, and services are synced. Once
some java_package options are "upstreamed", benchmarks and
interop-testing could also be synced.
2018-12-20 16:47:16 -07:00
ZHANG Dapeng 94e7339748
all: remove copies of gradle wrapper 2018-12-11 16:23:28 -08:00
ZHANG Dapeng dd1a7e19b9
buildscripts: remove jenkins-pre.bat 2018-12-11 10:36:58 -08:00
ZHANG Dapeng 3202fcc7d9
examples: move TLS example to a separate project 2018-12-06 12:03:25 -08:00
ZHANG Dapeng 00b4b8870b
examples: move alts bazel and rm alts from pom.xml 2018-12-03 17:12:30 -08:00
Eric Anderson 80883f0983 buildscripts: Reduce verbosity of protobuf builds
A substantial portion of our build logs are just compiling protobuf.
This reduces the noise while still keeping most of the useful
information.
2018-09-21 10:58:07 -07:00
Eric Gribkoff 03dab29e3a
buildscripts: run Android interop tests on Firebase (#4314) 2018-08-28 14:56:06 -07:00
zpencer b8f4c7a57a
buildscripts: disable gae jdk7 test (#4787)
The test has become very flakey recently. With GAE jdk7 going away
completely in early 2019, let's disable running the test in CI.
2018-08-23 17:14:49 -07:00
Eric Anderson 71e9841c80 travis.yml: Fix protobuf compilation caching
Wildcard apparently doesn't work... silently. Move versions into their
own directory so we can use a hard-coded string in the travis
configuration.
2018-08-17 17:27:12 -07:00
Eric Anderson 142851dcbb buildscripts: Use full SHA for protobuf download
Github used to produce tars with the full SHA in the folder name.
However, now they are providing tars with just as much SHA as was in the
initial request. This coincides with a change in its services:

Before:
Resolving github.com... 192.30.253.113, 192.30.253.112
Connecting to github.com|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/23357588/88f90e58-e598-11e7-9263-ef8b1fb345b8?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180817%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180817T190158Z&X-Amz-Expires=300&X-Amz-Signature=b684e1ead24a96764fd1f29ac5eb1df693d47c5fe86962088aee2ccfb8e4e72a&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dprotobuf-all-3.5.1.tar.gz&response-content-type=application%2Foctet-stream [following]

As of 2:00 PM PT today:
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/google/protobuf/tar.gz/61476b8e [following]

Note that repositories.bzl isn't impacted because it already specified
the full SHA.
2018-08-17 16:59:25 -07:00
Spencer Fang c3c810f5c3 buildscripts: Make kokoro script runnable from workstation
- Only cat /VERSION if it exists
- The host name is used as the version name, and if running locally
  the versions no longer necessarily start with 'kokoro'. Update the
  clean up command.
2018-08-16 15:03:03 -07:00
Spencer Fang 19f4491ce9 buildscripts: Clean up dummy default version check cmd
These statements can be simplified to be more readable.
2018-08-16 15:03:03 -07:00
Spencer Fang 9aa53b589d buildscripts,gae-interop-testing: do not promote versions by default
Only the dummy-default version should ever be promoted. Test versions
should have no traffic routed to it, so that deletions are
simpler. Versions receiving traffic can not be deleted in GAE.
2018-08-16 15:03:03 -07:00
Spencer Fang b821cf13b0 buildscripts,gae-interop-testing: Hardcode service name
This simplifies the kokoro script and makes things consistent between
the gradle script behavior and kokoro behavior.
2018-08-16 15:03:03 -07:00
Eric Anderson 8188a3eb92 gradle: Use config_loc in checkstyle
We previously passed a custom variable to the checkstyle configuration.
In Gradle 4.0 config_loc was added for the same purpose. The default
configDir is $projectDir/config/checkstyle which is different for each
project; we need to override it to always point to the root project.
2018-07-30 10:51:54 -07:00
Eric Anderson 4e276a52f3 buildscripts: Remove set -x from run_in_docker.sh
Printing the commands is pretty ugly output and not particularly
helpful, unless debugging the logic. If needing to debug, just add the
-x locally.
2018-07-20 14:08:07 -07:00
Eric Anderson 7bb88de264 buildscripts: remove grpc-java-releasing docker container
The buildscript was updated to pull in a fix to protoc-artifacts to
support TLS 1.2 (google/protobuf#4879), which was the only remaining
reason to have our own container.

Docker container building was split out into a separate build_docker.sh
so that people can call it blindly and get the container necessary for
run_in_docker.sh.
2018-07-11 13:04:30 -07:00
Eric Anderson 1887b93afd buildscripts: Hard-code PROTOBUF_VERSION in make_dependencies.sh
We always want to use a consistent version of protobuf; avoid the need
for the caller (which may be a person running the script) to specify the
version.
2018-07-11 10:13:09 -07:00
Eric Anderson 4879d6f6a8 buildscripts: Use "all" protobuf download to avoid ./autogen.sh
This cuts ~20 seconds from the build time and avoids the need to have
autoconf and libtool installed. This is the "normal" way to build
protobuf.
2018-07-11 10:12:33 -07:00
Eric Anderson 3a58a9999e Fix signatures in release process
We must not include signatures for maven-metadata.xml*, as the file is
a server-maintained index. Avoiding signing the .md5 and .sha1 avoids
signing unnecessary files (which double-excludes some
maven-metadata.xml* files).

We also swap to using ASCII armored output instead of the binary output,
as that's what is appropriate for "asc" files.

Fixes #4574
2018-06-20 11:29:32 -07:00
Eric Gribkoff 9a502ecf0b
buildscripts: correct Android apk size and dex diff output (#4522) 2018-06-05 12:50:39 -07:00
Eric Gribkoff 8fd762fb8e
cronet: use Cronet from Google's Maven repository (#4531) 2018-06-04 20:16:02 -07:00
Eric Gribkoff f3ef2850a9
buildscripts: fix Android apk/dex comparison (#4511) 2018-05-25 15:07:17 -07:00
zpencer b9df27b86f
buildscripts: add basic sanity check for android aar (#4448)
Just a very basic sanity check that the android artifact is not missing.
2018-05-07 15:43:08 -07:00
Eric Gribkoff b83312fb7b buildscripts: generate android artifact 2018-05-07 11:31:56 -07:00
zpencer c51a7749b1
buildscripts: assume artifacts are coalesced in upload_artifacts (#4439)
The `find` was unnecessary. Update the sanity check to reflect this
assumption.
2018-05-07 11:18:42 -07:00
zpencer 2404b25101
buildscripts: automate the release process (#4406)
These scripts set up the jobs needed to do one click releases.

List of changes:

Do not put files in mvn-artifacts/$ARCH. This makes the view uniform
across unix and windows and is easier to sign and upload.

A working curl is needed to build protobuf from source, so run yum
update.

run_in_docker.sh: use chmod to fix permisisons, do not set up user

linux_artifact builds 32 and 64 artifacts using unix.sh

add upload_artifacts cfg and script
2018-05-03 19:48:14 -07:00
Carl Mastrangelo 60a0b0c471
all: normalize copyright header 2018-05-03 14:55:21 -07:00
zpencer f8424f7b7b
examples: add kotlin and kotlin android examples (#4037)
Add kotlin version of HelloWorldClient and HelloWorldServer.
Add kotlin version of android example.
2018-05-01 14:54:34 -07:00
zpencer 1d80febbc0
buildscripts: make unix.sh and make_dependencies.sh arch aware (#4384)
ARCH can be '32' or '64'. If it is not set then default to '64'.

make_dependencies.sh should do the symlinking
2018-04-27 15:59:16 -07:00
zpencer c6c20e9491
buildscripts: add linux job that release_artifacts can dep on (#4392)
* buildscripts: add linux job that release_artifacts can dep on

* rename linux -> linux_artifacts

* fix script name
2018-04-25 15:54:15 -07:00
Carl Mastrangelo 720d4fab69
all: print out diff of methods in android APK 2018-04-25 15:53:56 -07:00
zpencer 218e944e16
buildscripts: initial kokoro config for auto releasing artifacts (#4391)
The script does nothing at the moment other than set up the kokoro
job.
2018-04-25 14:12:06 -07:00
zpencer 7cd2f5c3c0
buildscripts: sonatype uploader ussing CONF before it is set (#4388)
Set $CONF before we use it.
Also enable some defensive bash flags to catch errors.
Default value for USER and PASS
2018-04-25 12:54:49 -07:00
Eric Gribkoff ba86a86c77
android: add AndroidChannelBuilder (#4172) 2018-04-23 11:49:36 -07:00
Eric Anderson a47266ea4a kokoro: Avoid --include-build for Android
Since 4369e8cd the --include-build just opens us up to trouble with
accidentally building the protoc plugin. Since we're going to do a
./gradlew install anyway, let's just wait until after that point for
building cronet.

This sort of problem was experienced while developing #4369.
2018-04-21 00:03:44 -07:00
zpencer 48b7c62b43
buildscripts: always keep mvn artifacts (#4372)
After searching for "artifacts retention policy" in the
kokoro-users group, I learned that there's a default 90 day
retention policy already in place.
2018-04-20 15:19:48 -07:00
zpencer 9f92104b94
buildscripts: fix GAE dummy app deploy races (#4308)
This fixes a race where we can race with another deploy of the
'dummy' version.

Also, when we delete old versions, only delete versions beginning with
'kokoro'. This would make races even less likely because the
'dummy-default' should normally stick around forever.

This fixes this failure:
https://source.cloud.google.com/results/invocations/6b24aed4-7dc3-490a-87ea-c870d1435184/targets/grpc%2Fjava%2Fpresubmit%2Fgae-interop/log
2018-04-05 13:19:36 -07:00
Eric Anderson ef030abe6b java_grpc_library: re-add grpc-java repo to targets
They were removed in 137c74d1 since it was believed they were unnecessary.
However, since they are in a macro and not a rule, they are relative to the
caller, not their definition.

Added building the examples to the kokoro CI. Note that this means the examples
are built twice: once in grpc-java's build and once in their own (because it
has a WORKSPACE). Given that the Bazel build is our fastest build, this
slowdown won't probably be an issue.
2018-04-03 08:42:40 -07:00
zpencer 19a64cc336
buildscripts: build windows artifacts in kokoro (#4253)
We always build the artifacts, but only store them in placer via
kokoro if MVN_ARTIFACTS is set to a non empty value.

The 32 bit script runs tests as well as builds artifacts.
The 64 bit script only builds artifacts.
2018-03-22 17:56:59 -07:00
Eric Anderson 9c5e96e376
buildscripts: Cleaner Linux release Docker
This is a greatly simplified Docker container compared to that in
compiler/. We really want the docker image to just be a build
environment, and build the specific versions of dependencies as part of
our normal build (not docker build).

It also includes a helper script that lets you easily do build actions
with the docker environment, but into a checkout on the host. This can
dramatically reduce the pain in building from the docker container as
the source doesn't need to be checked out and only caches are lost
between invocations.
2018-03-13 09:38:21 -07:00
Eric Anderson ab92ff0819 buildscripts: Use ErrorActionPreference=stop with powershell
Otherwise when an exception is thrown powershell will still exit with 0.
This is similar to set -e, except exceptions wouldn't have a status code
of their own.

Another option would have been -ErrorAction Stop, but this need to be
passed to each command inside the powershell script, so would be
functionally similar to our "|| exit" style in the .bat.
2018-03-13 09:16:59 -07:00
Eric Anderson 4bd4cf4ec1 buildscripts: check errorlevel of error-prone commands in .bat files
There's no 'set -e' in BAT, so we get to do it manually. Wee!

Note that we need to use 'exit /b' instead of 'exit' in the scripts, as
otherwise it exits more than just the current script.
2018-03-13 09:16:59 -07:00
Eric Gribkoff 7fd22080e9
buildscripts: avoid unbound variable error (#4197) 2018-03-09 09:18:58 -08:00
Eric Gribkoff 4369e8cd9a
buildscripts: add Kokoro-based CI for Android APK stats (#3984) 2018-03-08 18:41:39 -08:00
Eric Anderson 9f677a991a buildscripts: Hard-code use of TLSv1.2 for github download
GitHub is now requiring TLS v1.2:
https://githubengineering.com/crypto-removal-notice/

Without this configuration, we see:
iwr : The request was aborted: Could not create SSL/TLS secure channel.
At line:1 char:5
+ & { iwr https://github.com/google/protobuf/archive/v3.5.1.zip -OutFile
protobuf. ...
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:Htt
  pWebRequest) [Invoke-WebRequest], WebException
   + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe
  ll.Commands.InvokeWebRequestCommand

Fixes #4120
2018-02-23 13:34:12 -08:00
zpencer a45d07bcb5
build: run unit tests on osx, ensure proto output included (#4081)
This adds back functionality that was accidentally dropped when
porting from travis:
- make sure generating protos will not lead to any uncomitted changes
- actually run the unit tests
2018-02-14 16:44:02 -08:00
zpencer fbc079d631
buildscripts: build macos artifacts in kokoro (#4066) 2018-02-14 12:53:11 -08:00
Eric Anderson f48a5097ef kokoro: Avoid getting cmake if it's installed
Kokoro already has cmake installed, so it's not necessary in that
environment. We still keep the old code around because it's helpful for
setting up new Windows environments in general.
2018-02-12 07:22:36 -08:00
Eric Anderson 850143f4a3 kokoro: Include kokoro image version in build logs
Otherwise there's no way to determine which version of the image was
used for the build.
2018-02-05 11:11:03 -08:00
Eric Anderson 5a31dff283 kokoro: Use --include-build in cronet
This avoids needing to build all of grpc as well as avoids having to
manually track which dependencies are necessary.

This cuts 1.5 minutes off the Cronet build time (to 3.5 minutes). But it
also reduces the amount being downloaded which should help with #3284.
2018-02-01 17:17:28 -08:00
zpencer 4e067eb651
gae: set -e was clobbering $? (#3999) 2018-01-25 17:19:57 -08:00
Eric Gribkoff 8ecef08d33 buildscripts: add Kokoro-based CI for gRPC Cronet 2018-01-16 09:16:04 -08:00
Eric Anderson 0853f3563f buildscripts: Add Kokoro-based Bazel CI 2018-01-11 10:28:21 -08:00
Eric Anderson 80e61d2589 clear JAVA_HOME 2018-01-11 09:27:33 -08:00
Eric Anderson 396cf4d901 buildscripts: Avoid Java 9 for Windows CI
Our build isn't happy on Java 9 yet.
2018-01-11 09:27:33 -08:00
Kun Zhang 86d64122e5
Upgrade to Protobuf 3.5.1 and Protoc 3.5.1-1. (#3921)
protoc-3.5.0-linux-x86_64 introduced GLIBC_2.14 dependency and broke
gRPC release process (https://github.com/google/protobuf/issues/4138).
3.5.1-1 is the proper re-build.
2018-01-05 16:40:20 -08:00
Spencer Fang 610066896f buildscripts: filter in only the kokoro java project 2018-01-04 10:57:35 -08:00
Spencer Fang d86647aebd buildscripts: automatically delete versions older than 1hour, always succeed if reach end 2018-01-04 10:57:35 -08:00
Spencer Fang 692cffe763 buildscripts: automatically delete versions older than 1day 2018-01-04 10:57:35 -08:00
Spencer Fang cdc480d353 buildscripts: fix error using cmd output rather than $? 2018-01-02 14:21:40 -08:00
Eric Anderson 4f4cedf92a
build: Support uploading releases separately from building 2017-12-14 16:51:14 -08:00
Eric Anderson b9278e7011 buildscripts: Remove dependency on 7za
We already depend on Powershell, so this should make it easier to use the
script with less set-up.
2017-12-06 16:31:00 -08:00
Carl Mastrangelo aee5fc4176
all: update to proto 3.5.0 2017-11-30 11:50:19 -08:00
zpencer d080bae2a1
gae-interop-testing: kokoro script to run interop tests in GAE (#3731) 2017-11-29 11:20:15 -08:00
zpencer 53c135a48f
buildscripts: set -u, set -o pipefail (#3629)
Be even more defensive with our shell scripts, to avoid future mistakes.
2017-10-27 15:31:30 -07:00
zpencer d840227165
buildscripts: make the script fail for the right reason when using set -e (#3628) 2017-10-27 08:54:08 -07:00
zpencer 8970c3a7c1 kokoro: port .travis.yaml to kokoro for mac osx (#3608) 2017-10-26 14:26:35 -07:00
zpencer 2eb47eaab7 kokoro: Initial config files for macos (#3599) 2017-10-23 16:10:11 -07:00
zpencer 11272f4780 kokoro: add a placeholder script for GAE testing (#3590) 2017-10-19 16:26:52 -07:00
Carl Mastrangelo 24ff2748b7 all: update to proto 3.4.0 2017-08-25 11:25:36 -07:00
Matt Kwong 5cf469ae0b all: make Kokoro show actual Gradle results (#3208) 2017-07-10 17:26:33 -07:00
Eric Anderson e6d0062d20 Move checkstyle config to buildscripts/
Checkstyle configuration is just noise in the root directory. Neither
users nor developers need to look at it often.
2017-07-07 17:11:40 -07:00
Matt Kwong 16285dc634 Make Kokoro logs less noisy (#3110) 2017-06-30 13:53:52 -07:00
Matt Kwong dda8f547e7 Add Windows build and config files for Kokoro (#3096) 2017-06-19 10:47:23 -07:00
Carl Mastrangelo 0fe2c5cca6 all: bump to proto 3.3.1 2017-05-23 17:04:51 -07:00
Carl Mastrangelo b0323ac22c all: update to protobuf 3.2.0 2017-02-07 09:47:15 -08:00
Eric Anderson c38611a230 Bump protobuf to 3.1.0
Also removed warnings about protoc version matching runtime, since this
is no longer supposed to be a problem (starting with 3.0.0-beta-4) and
all our tests ran fine when using protoc 3.0.2 with protobuf runtime
3.1.0.

Fixes #2316
2016-11-01 14:52:56 -07:00
Eric Anderson b1d72e5c3e all: Bump protobuf to 3.0.2, to fix protoc in CI
protoc no longer builds in 3.0.0 because auto-download of the gmock zip
now fails. 3.0.2 has a fix to autogen:
bba446bbf2

All that was strictly necessary was to update .travis.yml and
buildscripts/, but it helps our sanity to keep the rest of the protobuf
versions in sync. Lite is left on its existing version, because it did
not see a bump of neither the java library nor the protoc plugin.
2016-09-29 10:25:31 -07:00
Eric Anderson ff7fb5ff52 jenkins: Fix Windows build to specify Studio version
If there are multiple versions available, cmake won't choose the Visual
Studio version selected by vsvars. So we have to explicitly specify the
generator to use.

This allows grpc-java to run on the shared Windows workers instead of
its own specialized instance.
2016-08-08 09:12:15 -07:00
Eric Anderson 5e1e88357c Update protobuf to 3.0.0
Fixes #2086
2016-07-29 09:31:15 -07:00
Eric Anderson 3c03eb79ec travis,jenkins: Fail build if javac emits warnings 2016-07-10 12:01:07 -07:00
Eric Anderson dbef1af29a Bump protobuf dependency to 3.0.0-beta-3
This allows us to play with zero-copy and proto3 support for lite.
Unfortunately, it introduced some warnings, so deprecated warnings are
now ignored for benchmarks and interop-testing.
2016-06-28 08:58:13 -07:00
Eric Anderson 80f73a4a16 Allow cache to be shared between Linux and OS X
It doesn't appear that OS X is saving its cache yet, so this may not be
needed in the future, but for now we need OS X to avoid attempting to
use the Linux binaries.
2016-04-27 19:35:06 -07:00
nmittler 1dce8df077 Switching to netty-tcnative-boringssl-static 2016-02-16 13:11:45 -08:00