Add timeout to all builds and set the proper values for the existing
timeouts
refs DE-307
flag = none
test-plan:
- Force the build to timeout and check that jenkins fails the build
Change-Id: I2e3ae36f828da1bc047bf5a2cfbc79c167b84d5f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256171
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Add a commitFlag that allows the user to skip CI
refs DE-192
flag = none
[canvas-builds-refspec=618e43174bc58f54905421e18b212d96f9eb0b95]
test plan:
- Ensure the build is skipped using the flag
- Ensure a code-review -2 is assigned if the skip ci flag is enabled
- Ensure the build is executed and passes when the flag is removed
Change-Id: I5c57d9959e4650ccd42578672f4df38ca9abce34
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256070
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Use canvas-builds to create prefix for cache images
refs DE-435
flag = none
[canvas-builds-refspec=d54d1f4dda757fbbb0372ad26defd8104d838e47]
Change-Id: Id1c6e29cb025f4a660bf891faa95df13f2978c33
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/255977
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Add a ruby 2.7 build triggerable by a comment on gerrit
refs DE-387
flag = none
test-plan:
- Build passess
- Local dev builds
- Ruby 2.7 build uses the ruby2.7-passenger image for
building the application
Change-Id: I512df9344dbd77e4c592270ef260325f95d8f666
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/255797
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-412
In order to cache the webpack-builder image by MD5SUM, we need to split it into 2 pieces, yarn-runner / webpack-builder so that changes to the packages directory don’t need to reinstall all of yarn.
[build-registry-path=jenkins/canvas-lms/de-412-pt-2a]
[change-merged]
Change-Id: Ib90920d0a8330ebee0365d530c3d4a0744da779a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/255044
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-413
Some users want to be able to load the built JS image locally for debugging purposes. We have 3 JS jobs that run in parallel and they all build the image separately. Create a 4th JS job that only runs if this image is requested in order to only upload the image once.
[upload-js-debug-image]
Change-Id: Iaa63971dcdf7bbae0c201a5fd0c8ffc319873a22
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254871
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-407
We want to ensure that the pre-merge build cache for new commits is available as soon as possible. To that end, we allow the webpack cache to attempt to use an available pre-merge built image and tag it as the post-merge image.
Also, rename the WEBPACK_CACHE_*_SCOPE variables to CACHE_*_SCOPE in anticipation of ruby / yarn usage.
Test Plan:
1. Make a webpack change
2. Build the PS with a unique build-registry-path
3. Add the change-merged directive to that PS and ensure that the pre-merge build image cache pulls the pre-existing tag, and retags it as the generally available cache image.
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-407]
Change-Id: Ibf86f94c2c91a80b0a0851a00cdea0db2822e4eb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254644
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-407
Now that build arguments are included in the cache ID, we don’t need to have a separate prefix anymore for each type of image.
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-407]
Change-Id: I940bf09b856dc57a9256633fe3a96fac98219c6c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254625
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-320
Test Plan:
1. Ensure test counts are consistent between current master and this change.
[canvas-builds-refspec=544a8f8710c148035fd39b1118e118ae963f520f]
Change-Id: I56b9ee05321528aef96c92e090cd1b484a525162
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252805
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-398
Currently, when the webpack-cache image is built, pre-merge builds tag the image with the same format as the post-merge default cache. This causes debugging to be difficult and also prevents the image from being cleaned up later. Fix this by prepending a suffix to each type of image, branch name for post-merge and patchset number for pre-merge.
When we do the above, we can also cache patchset-specific images for free. This will allow rebuilds to use a fully cached docker image if the webpack-related files have not changed.
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-398]
[canvas-builds-refspec=53a5214f29ec16da5ea3aea28d77d0286b1c7deb]
Change-Id: I09e0c8792c688c899ca32b934e86be65202b4102
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254068
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
[change-merged]
[build-registry-path=jenkins/canvas-lms/ruby-runner]
Test Plan:
1. Ensure that changing gems updates the whole image.
2. Ensure that changing npm updates the webpack builder image.
3. Ensure that changing webpack files are reflected in the final image.
4. Ensure that changing packages files are reflected in the final image.
5. Ensure that there are no speed regressions in fully-cached cases.
6. Ensure that caching works as expected.
Change-Id: Icef77195ff912bc0628f27eeda6780f262a8908a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252565
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
move the logic related to changed files in commit check to
canvas-builds
refs DE-366
flag = none
test plan:
- Build passes
- Main Build detect if there was a change in Jenkins file in HEAD
- Local Docker dev Build is triggered if a local dev file changes
- JENKINSFILE_REFSPEC is setted if a Jenkinfiles changes
Change-Id: I497cae9654e1b71b2de5ec3325b9577b30cfeeba
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253555
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-379
flag = none
test-plan:
- Build passes
- Simulate build over 100 commits behind fails
- Forced merge-conflict fails on rebase
- A PS on dev branch rebases on dev and then on master
Change-Id: Id715ad837221f91f4c2931c14b9b3332707c7120
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253092
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
we want to always checkout master code for testing on jenkins, unless
there are changes to our build scripts then pull the gerrit refspec.
flag = none
Test Plan:
- Normal Jenkins build passes.
- changes to docker-compose.new-jeknins* causes sub builds
to checkout code from gerrit refspec.
- changes to any file in build/new-jenkins/ causes sub
builds to checkout code from gerrit refspec.
- Normal build with no changes to Jenkinsfile*, build/new-jenkins/,
or docker-compose.new-jenkins* sub builds checkout master.
Change-Id: If1224a44b52b00f0f056b9552cd562809fc7eaa8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252999
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs DE-376
With the ruby-runner change that is coming, the final image will no longer include the node_modules/ assets, so we have to checkout the WEBPACK_BUILDER separately and copy in the source from the final PATCHSET_TAG.
Change-Id: Ic3c048500e00858bfa39244b308112cbc721783d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252666
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-376
In order to move our JS jobs to using the WEBPACK_BUILDER_TAG, we need to tag the specific image used for that patchset.
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-376-tag-webpack-builder]
Change-Id: I03b7754b83bd273994909b5009435ebc6fe3ceb9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252987
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-376
With the ruby-runner change that is coming, the final image will no longer include the node_modules/ assets, so we have to checkout the WEBPACK_BUILDER separately and copy in the source from the final PATCHSET_TAG.
Change-Id: Ide3885f41d7ee21f4034331c204c2cafdc76aa28
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252922
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-376
Prepare for the ruby-runner change. Prevent the patchset-specific layer(s) from being included in the cache images, since they are virtually guaranteed to be invalidated.
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-376]
Change-Id: Ib97fd7f657830d2ea133159730c3c03d6aa7d919
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252878
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
when using stable branches, pull all plugins from same branch.
Test Plan:
- Jenkins build passes
- Jenkins build from stable branch pulls stable branch plugins
- Jenkins plugin build from branch pull canvas and other plugins
from same branch.
- test with canvas-lms-refspec commit flag and without
Change-Id: I07d62f9c75413fe2a0c035b43a14634baa814eef
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252250
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Moving logic related to the creation of a stage in Jenkinsfile to
canvas-builds.
refs DE-334
flag = none
test-plan:
- Build passes using the new makeFromJob function
- Build behaves as expected that is the same way the current
Jenkinsfile works
- Ensure CDC works forcing a migration
- Ensure FSC works forcing failures
Change-Id: I70b5e68e03166c5a34bf7542c10e8751e15981d6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250770
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
[canvas-builds-refspec=de97029ae53f268357222cbdea908873f5c2f7d7]
dev branches will rebase with their head, then rebase with master.
stable branches no rebase.
any other branches will follow the stable branch workflow.
flag = none
closes: DE-18
Test Plan:
- Normal build passes
- dev branch build rebases correctly
- image tags are working with branches, replacing
forward slash with underscore.
Change-Id: Ibf3f4b97c42b0aea5d49f14bb498a09dae7c091f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251756
QA-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
refs DE-361
Test Plan:
1. Normal build fails for changes to Jenkinsfile
2. Normal build fails for changes to Jenkinsfile.*
3. Jenkinsfile build works for changes to Jenkinsfile via manual trigger
4. Jenkinsfile build works for changes to Jenkinsfile.* via manual trigger
Change-Id: Ie61120a4b1798b39290ba879d28a4614ab57450e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250679
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
remove all the logic around successes lib from canvas-builds
refs DE-355
flag = none
test-plan:
- Build passes if we use a canvas-builds version that does not
include successes lib
- Build rerun all stage when retriggered from a forced failure
- Build passess
Change-Id: I0127f9992a2a307a953f6cbeac5450a17535801e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251795
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Aaron Ogata