refs DE-656
ExtendedStage supports more alternative statuses than FAILED / SUCCESS now, such as NOT_BUILT, ABORTED_USER and ABORTED_PATCHSET. All of these need to be treated as FAILED conditions.
Test Plan
1. All patchsets start appearing in #canvas_builds-noisy
Change-Id: I0eae928a7d232c730e7c552336c216c09d62bfef
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263797
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Product-Review: Ryan Norton <rnorton@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
refs DE-558, DE-582
[use-kubernetes]
[canvas-builds-refspec=24d3d83e8034ac6ad5380936498e1dcf01ab50eb]
modify the existing nodeRequirements to add podTemplate and
container definitions. initially, the podTemplate to be used
is docker-in-docker. the "use-kubernetes" flag must be set
as well.
test plan:
- build executes on k8s nodes in EKS when "use-kubernetes"
flag is set.
- build executes on EC2 when "use-kubernetes" flag is not set.
Change-Id: I13c68e5420bbf414532518eaf69ac82f9310bfbf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263257
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-630
In order to start auto-killing bad nodes, we’ll need a list of exceptions that represent that state. Start collecting them in #canvas_builds-noisy.
Test Plan
1. Common failure cases don’t report to Slack
[canvas-builds-refspec=e4b78afbb94053b352b1ad7ef514ac1efdcf1558]
Change-Id: I93e57cfe49f4ad190769a101765cd870a6d16ca5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263553
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-628
Test Plan
1. Ensure that code / webpack always run
2. Ensure that yarn linters run only when package.json or yarn.lock changes
3. Ensure that build summary report works when each stage fails
[canvas-builds-refspec=f2c74ae95319648ac9fe19f306d7067adda9c534]
Change-Id: I64eb9f794e00737b2f0b87a1d962a4aaaeb174d2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263365
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-628
In order to improve the build summary report UX and also to increase compatibility with podTemplate / container, split the setup / tear down pieces of the Linters stage to prepare for moving each piece to its own Jenkins stage.
Test Plan:
1. All linters run and fail if error is found
2. Linters post status to gerrit
3. Plugin build Linters works
Change-Id: I4c4769eed80f201532edc34907f2428ceb465773
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263323
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-633
When a sub-job is cancelled, it won’t return a RunWrapper, so it can be null here.
Change-Id: Ia49b81268d6548c58ce9ba35d38175c5fd76705d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263295
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-558, DE-582
[canvas-builds-refspec=f9c80c0ac4b80de60ac50633523e30d5c2eb28b1]
modify the existing nodeRequirements call to pass a Map containing
the label as well as the possibility of container or podTemplate
params. this change only modifies the existing nodeRequirements to
continue using EC2 agents, not EKS.
test plan:
- builds still work as expected acquiring nodes
Change-Id: I0458a13a4c85e37aeb2f5dc56347e27101705646
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263208
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
refs DE-619
Merge the current logic for handling the main build failures with the logic for sub-jobs to reduce complexity and ensure consistent handling.
Test Plan:
1. Ensure sub-job test results appear
2. Ensure rspec / selenium failure stages appear
3. Ensure sub-job failure stages appear
4. Ensure parent stages do not appear
[canvas-builds-refspec=6e70b5b572e9297f1cd06020ec61a551e7c0d323]
Change-Id: I7410cec5340714b3617a310f83549011a2a0ce0a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262979
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-338
EKS requires that we stop using on-demand instances for any of our build nodes. Now that the pre-requisite work is done to allow this change, actually switch it over.
Test Plan
1. Spot instance termination of primary node works when Linters job is running
Change-Id: Idac911c2c8266e30dc9cb8f0b7478fc09c8aa543
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263160
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-628
Test Plan:
1. Build summary report works for sub-jobs and normal stages
2. Sub-jobs run cleanupFn
[canvas-builds-refspec=82ccf7ae185871e45c658f225cc664ef05cda74d]
Change-Id: I9d4476b75e6488bcc388a89e9f7cf87150d0ae6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262951
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-621
Make the stage configuration aware of its node requirements for future EKS support and allowing nospot to be removed.
Test Plan:
1. RSpec / Selenium build works
2. Contract Tests works
3. JS works
4. FSC works
5. Vendored Gems works
[canvas-builds-refspec=480f3570a6a6ebee5cedaecbb3a089476a008215]
Change-Id: I6cb63df20abe8c1f913db4f1b21bb01bd4d30312
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262898
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-624
Jenkins CPS transformation doesn’t correctly transform the Closure inside of the Expando object, but we can use a Map instead that has the same functionality (the property can be accessed using the dot operator).
Test Plan:
1. Build summary report works
Change-Id: Ic3446afa20f6c6ce5aac95e5e0cda06edfa39ef5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262844
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-624
As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.
Test Plan:
1. All stages execute correctly
[canvas-builds-refspec=07f1a75ee23f8f6a9a1bbe062316373b761e7a30]
Change-Id: Ica11b78ebead5c9239dea112677a077654c7090d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262814
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-338
Test Plan:
1. Build succeeds when all stages pass, and uploads a summary report.
2. Build fails when any stage fails, and uploads a summary report.
Change-Id: I98d9ce4507ed805caad4fe4857aa9ecef9aceaf9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262780
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-620
It can be helpful for monitoring builds after potentially breaking changes are made to have a list of build success / failure in one place.
Change-Id: I5b06d3221e8292740fdcccdb4e1633f9ac30177a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262764
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
Eliminate the need for the withEnv() indentation by consistently naming all of these variables.
Test Plan:
1. Migrations caching works correctly
Change-Id: Iaef43c4e4278878e17b90e2190dc0d7a0a42cb26
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
After we started moving stages out of the nospot block, we forgot to move the post-actions function out of that block as well. This results in postFn() being called too early.
Test Plan:
1. Build summary report actually captures all stages.
Change-Id: I743827c4f35682e8887918c9041ed833e51ad2f3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262712
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
Now that stages that don’t start with the same name as the waiting stage exist, we need to wrap the waiting stages with obeysAllowStages(false) so that they execute correctly.
Test Plan:
1. allow-stages works with non-JS jobs
[canvas-builds-refspec=871c7a9eb43015c9eb3faccca73742ebb4169166]
Change-Id: I52f1cf1c9778c36030d4582abcebf2ee03577122
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262598
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
This functionality has been replaced by protectedNode() only calling this clean-up function when a node has been acquired.
Test Plan:
1. main Jenkins passes
Change-Id: I760fd3aaecf4f626b5dc4c65401483f95cb65923
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262591
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
Nospot removal requires all non-build stages to not be wrapped in the protectedNode() function.
Test Plan:
1. Local Docker Dev triggers correctly when files change.
Change-Id: I986a85161d567c31606b73553c420cf63bdec455
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262571
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-617
In order to properly split the stages for nospot removal, none of conditional jobs in Parallel Run Tests should directly depend on a heavyweight executor to know if they need to run.
Test Plan:
1. CDC Schema Check triggers when migrations change.
2. FSC triggers when specs change.
3. Local Docker Dev triggers when related files change.
[canvas-builds-refspec=3828a4bcd9f172fb6cec7046742cb678e3e489bf]
Change-Id: I578e4665d382aa27f03932d172604b20a8a1497d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262551
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-617
extendedStage() has functionality that allows a stage to be required or not. If it is required, the stage is run. If it is not required, it is not run unless explicitly allowed via the allow-stages directive.
Test Plan
1. All pre-merge stages run correctly
1. All post-merge stages run correctly
[canvas-builds-refspec=ee0987058d3d29d8130904b126615a1ab20f9ca2]
Change-Id: Ie9709b50a2a6f75e5687972c49bfe488ee3b6be1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262540
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Move all the logic related to the setup stage to its own
groovy file.
refs DE-615
flag = none
Test-plan:
- Setup stage works the same way
- Main build passes
Change-Id: I32771f8a6262f3dad905ea71d8774cbdf96aac57
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262525
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Move all the logic related to the dependency check stage to its own
groovy file.
refs DE-615
flag = none
Test-plan:
- Dependency Check works simulating post-merge
Change-Id: I2164ab028b885cea0f8a72c6c6fb1d09360403c8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262514
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Move all the logic related to the linter stage to its own
groovy file.
refs DE-615
flag = none
Test-plan:
- Linters works
Change-Id: I4b15b59bf7d76b16ce5f9773d8be4565f08849c0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262511
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
In order to make the Jenkinsfile more readable move the rebase logic
to a self groovy file rebaseStage. This will also allow to create
unit test against it
[canvas-builds-refspec=fa26046c2b1382a956c3891debcd54eac7ed4166]
refs DE-615
flag = none
test-plan:
- Rebase stage should passes and behaves the same way
Change-Id: I7649346572b3ad6a0684dbdc75ae6ba4682fe5e0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262488
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-615
In order to abstract our stage implementations into the local shared library in a clean way, we need to prevent referencing isPatchsetPublishable() by the Build Docker Image job.
Test Plan:
1. Ensure that the EXTERNAL_TAG is uploaded after a successful build.
Change-Id: I831d19f51a73f83849a6d173a03744d184fc6798
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262490
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-611
As part of our transition to chained methods to configure stages, move this pattern over as well for completeness.
Test Plan:
1. Corresponding canvas-lms build passes when directive is used.
[canvas-builds-refspec=494be81ae38b517ba87e44ac3d1da0962d4141d6]
Change-Id: Ib75426bf01633bd458312e5cf6f076feb7b8aafc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262476
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-602
We need to reduce the different permutations of extendedStage() that live outside of that step. In order to do this, a handler can be registered that provides a pre-defined interface of functions that can be called.
Test Plan:
1. Jenkins main build passes.
[canvas-builds-refspec=0836fe0779bf81547ed1ea1c063ba806dc2a35e5]
Change-Id: I0cc268d09f10459b8417b0386383435b2ac74b74
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262402
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
move rebaseHelper function into its own groovy file.
refs DE-587
flag = none
test-plan:
- rebase stage works as expected
Change-Id: I7919f849b2acb4294c666fc896b201fc5ddc8b0a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261653
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
move rspec and distribution into a library folder that
can be loaded as local shared lib to avoid the load code
in the jenkins files.
refs DE-555
flag = none
[canvas-builds-refspec=d4cf97caed1edf2c9c91c48460e94f99e58a4d0a]
Change-Id: Id947a389d22f04126a8654bd5e4957433cddd330
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261289
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Aaron Ogata