refs DE-338
EKS requires that we stop using on-demand instances for any of our build nodes. Now that the pre-requisite work is done to allow this change, actually switch it over.
Test Plan
1. Spot instance termination of primary node works when Linters job is running
Change-Id: Idac911c2c8266e30dc9cb8f0b7478fc09c8aa543
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263160
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-628
Test Plan:
1. Build summary report works for sub-jobs and normal stages
2. Sub-jobs run cleanupFn
[canvas-builds-refspec=82ccf7ae185871e45c658f225cc664ef05cda74d]
Change-Id: I9d4476b75e6488bcc388a89e9f7cf87150d0ae6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262951
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-621
Make the stage configuration aware of its node requirements for future EKS support and allowing nospot to be removed.
Test Plan:
1. RSpec / Selenium build works
2. Contract Tests works
3. JS works
4. FSC works
5. Vendored Gems works
[canvas-builds-refspec=480f3570a6a6ebee5cedaecbb3a089476a008215]
Change-Id: I6cb63df20abe8c1f913db4f1b21bb01bd4d30312
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262898
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-624
Jenkins CPS transformation doesn’t correctly transform the Closure inside of the Expando object, but we can use a Map instead that has the same functionality (the property can be accessed using the dot operator).
Test Plan:
1. Build summary report works
Change-Id: Ic3446afa20f6c6ce5aac95e5e0cda06edfa39ef5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262844
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-624
As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.
Test Plan:
1. All stages execute correctly
[canvas-builds-refspec=07f1a75ee23f8f6a9a1bbe062316373b761e7a30]
Change-Id: Ica11b78ebead5c9239dea112677a077654c7090d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262814
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-338
Test Plan:
1. Build succeeds when all stages pass, and uploads a summary report.
2. Build fails when any stage fails, and uploads a summary report.
Change-Id: I98d9ce4507ed805caad4fe4857aa9ecef9aceaf9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262780
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-620
It can be helpful for monitoring builds after potentially breaking changes are made to have a list of build success / failure in one place.
Change-Id: I5b06d3221e8292740fdcccdb4e1633f9ac30177a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262764
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
Eliminate the need for the withEnv() indentation by consistently naming all of these variables.
Test Plan:
1. Migrations caching works correctly
Change-Id: Iaef43c4e4278878e17b90e2190dc0d7a0a42cb26
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
After we started moving stages out of the nospot block, we forgot to move the post-actions function out of that block as well. This results in postFn() being called too early.
Test Plan:
1. Build summary report actually captures all stages.
Change-Id: I743827c4f35682e8887918c9041ed833e51ad2f3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262712
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
Now that stages that don’t start with the same name as the waiting stage exist, we need to wrap the waiting stages with obeysAllowStages(false) so that they execute correctly.
Test Plan:
1. allow-stages works with non-JS jobs
[canvas-builds-refspec=871c7a9eb43015c9eb3faccca73742ebb4169166]
Change-Id: I52f1cf1c9778c36030d4582abcebf2ee03577122
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262598
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
This functionality has been replaced by protectedNode() only calling this clean-up function when a node has been acquired.
Test Plan:
1. main Jenkins passes
Change-Id: I760fd3aaecf4f626b5dc4c65401483f95cb65923
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262591
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-338
Nospot removal requires all non-build stages to not be wrapped in the protectedNode() function.
Test Plan:
1. Local Docker Dev triggers correctly when files change.
Change-Id: I986a85161d567c31606b73553c420cf63bdec455
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262571
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-617
In order to properly split the stages for nospot removal, none of conditional jobs in Parallel Run Tests should directly depend on a heavyweight executor to know if they need to run.
Test Plan:
1. CDC Schema Check triggers when migrations change.
2. FSC triggers when specs change.
3. Local Docker Dev triggers when related files change.
[canvas-builds-refspec=3828a4bcd9f172fb6cec7046742cb678e3e489bf]
Change-Id: I578e4665d382aa27f03932d172604b20a8a1497d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262551
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-617
extendedStage() has functionality that allows a stage to be required or not. If it is required, the stage is run. If it is not required, it is not run unless explicitly allowed via the allow-stages directive.
Test Plan
1. All pre-merge stages run correctly
1. All post-merge stages run correctly
[canvas-builds-refspec=ee0987058d3d29d8130904b126615a1ab20f9ca2]
Change-Id: Ie9709b50a2a6f75e5687972c49bfe488ee3b6be1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262540
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Move all the logic related to the setup stage to its own
groovy file.
refs DE-615
flag = none
Test-plan:
- Setup stage works the same way
- Main build passes
Change-Id: I32771f8a6262f3dad905ea71d8774cbdf96aac57
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262525
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Move all the logic related to the dependency check stage to its own
groovy file.
refs DE-615
flag = none
Test-plan:
- Dependency Check works simulating post-merge
Change-Id: I2164ab028b885cea0f8a72c6c6fb1d09360403c8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262514
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Move all the logic related to the linter stage to its own
groovy file.
refs DE-615
flag = none
Test-plan:
- Linters works
Change-Id: I4b15b59bf7d76b16ce5f9773d8be4565f08849c0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262511
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
In order to make the Jenkinsfile more readable move the rebase logic
to a self groovy file rebaseStage. This will also allow to create
unit test against it
[canvas-builds-refspec=fa26046c2b1382a956c3891debcd54eac7ed4166]
refs DE-615
flag = none
test-plan:
- Rebase stage should passes and behaves the same way
Change-Id: I7649346572b3ad6a0684dbdc75ae6ba4682fe5e0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262488
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-615
In order to abstract our stage implementations into the local shared library in a clean way, we need to prevent referencing isPatchsetPublishable() by the Build Docker Image job.
Test Plan:
1. Ensure that the EXTERNAL_TAG is uploaded after a successful build.
Change-Id: I831d19f51a73f83849a6d173a03744d184fc6798
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262490
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-611
As part of our transition to chained methods to configure stages, move this pattern over as well for completeness.
Test Plan:
1. Corresponding canvas-lms build passes when directive is used.
[canvas-builds-refspec=494be81ae38b517ba87e44ac3d1da0962d4141d6]
Change-Id: Ib75426bf01633bd458312e5cf6f076feb7b8aafc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262476
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-602
We need to reduce the different permutations of extendedStage() that live outside of that step. In order to do this, a handler can be registered that provides a pre-defined interface of functions that can be called.
Test Plan:
1. Jenkins main build passes.
[canvas-builds-refspec=0836fe0779bf81547ed1ea1c063ba806dc2a35e5]
Change-Id: I0cc268d09f10459b8417b0386383435b2ac74b74
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262402
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
move rebaseHelper function into its own groovy file.
refs DE-587
flag = none
test-plan:
- rebase stage works as expected
Change-Id: I7919f849b2acb4294c666fc896b201fc5ddc8b0a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261653
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
move rspec and distribution into a library folder that
can be loaded as local shared lib to avoid the load code
in the jenkins files.
refs DE-555
flag = none
[canvas-builds-refspec=d4cf97caed1edf2c9c91c48460e94f99e58a4d0a]
Change-Id: Id947a389d22f04126a8654bd5e4957433cddd330
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261289
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Aaron Ogata