Switches from standalone containers to explicit node+hub config
Selenium 4 has some differences in handling stale elements that we
should be aware of moving forward
closes OUT-4988
flag=none
[skip-stages=Flakey Spec Catcher]
Test-plan:
- make sure screenshots can happen for failures
- retrigger a few times and make sure things pass
- verify build summaries are intact
- verify FSC can still run seleniums
- verify local selenium running still works
- firefox / chrome / edge where applicable
- verify docker selenium running still works
- firefox / chrome / edge where applicable
Change-Id: I8f2fe5a34d712b5ccd7191bae7a9aeeb6f1f473d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284811
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
[ignore-stage-results=Flakey Spec Catcher]
update chrome version to run tests against. A few small
fixes to work with new chromedriver.
closes: DE-930
Test Plan:
-Jenkins passes
-FSC build passes
-Run test local docker dev setup
-Run test local dev setup
Change-Id: Ie2d5dd918f85cbcc01d0f8b572efa8aab71c1803
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279959
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
closes OUT-4802
flag=none
Test-plan:
- test with a custom_reports PS referencing this
- see comments for expectations of FSC re-runs, compare against
actual
Change-Id: I94ead7b61e37512231b558b16b57a8bfb2cff4c8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276101
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
closes OUT-4802
flag=none
Test-plan:
- in PS1 builds,
- examine changed specs and match it up with FSC queued up tests
- in Jenkinsfile.selenium.flakey_spec_catcher, examine output, the new
"raw result from catcher" and "raw result from splitter" should output
similar lists
- examine any differences and whether they are noteworthy or not
- in PS2 builds,
- verify that FSC sub-build still functions after removing splitter
Change-Id: Icae697f1cedca8ab9a775a1f394891d4ca2007a3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275765
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
Test-plan:
- view PS4, see that changes were made that would have caused
shared examples to be queued up directly in FSC rather than
from within their contexts
- k5_dashboard_student_spec.rb:77 should be run by FSC
but
- k5_announcements_shared_examples.rb:193 should no longer be
added
Change-Id: If909647c5efe10137b734f47a074365bcef3c944
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274596
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
refs DE-719
withStarlordDockerLogin was needed for EKS, but with EC2
no docker login is needed.
test plan:
- EC2 builds continue to work as expected
Change-Id: Icbd3f28bf2ec6ec52c429c1eaf8fc6ce7bddd46b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267714
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Fix linter issues related to variable name, space after closing bracket.
refs DE-690
flag = none
test-plan:
- Build passes
- QA build with spec change executes fsc stage
Change-Id: I6f985fb3dfa8627703a27325b1eb25fae4585505
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265188
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-467
it is no longer possible to spin up multiple docker containers
ona single rspec or selenium agent when running tests. parallelization
is now exclusively done through multple rspec processes on
a single container.
test plan:
- setting docker_processes ENV variable has no effect
- settting rspec_processes still runs multiple test threads
- tests continue to run as expected given that docker_processes has
been set to 1 for multiple months.
Change-Id: I604d953a71ad220373b6a71d558f46bff29f820e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263701
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs DE-467
this change will require a followup to remove
DOCKER_PROCESSES.
test plan:
-run FSC with a changed spec.
-FSC runs the modified spec 30 times, 10 times in each thread.
-run FSC with multiple chnaged specs
Change-Id: I640c7b87e439db3c7efb42ac4e7f81e10dcbc5c9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264703
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-467
this change will require a followup to remove
DOCKER_PROCESSES.
test plan:
-run FSC with a changed spec.
-FSC runs the modified spec 30 times, 10 times in each thread.
Change-Id: I26755f6b9c3bf63267eb9d1ac648901d67ee41df
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264129
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs DE-558, DE-582
[use-kubernetes]
[canvas-builds-refspec=24d3d83e8034ac6ad5380936498e1dcf01ab50eb]
modify the existing nodeRequirements to add podTemplate and
container definitions. initially, the podTemplate to be used
is docker-in-docker. the "use-kubernetes" flag must be set
as well.
test plan:
- build executes on k8s nodes in EKS when "use-kubernetes"
flag is set.
- build executes on EC2 when "use-kubernetes" flag is not set.
Change-Id: I13c68e5420bbf414532518eaf69ac82f9310bfbf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263257
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-558, DE-582
[canvas-builds-refspec=f9c80c0ac4b80de60ac50633523e30d5c2eb28b1]
modify the existing nodeRequirements call to pass a Map containing
the label as well as the possibility of container or podTemplate
params. this change only modifies the existing nodeRequirements to
continue using EC2 agents, not EKS.
test plan:
- builds still work as expected acquiring nodes
Change-Id: I0458a13a4c85e37aeb2f5dc56347e27101705646
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263208
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
refs DE-338
The protectedNode() function now handles automatic cleanup before / after a node is used, so much of the code here can be removed.
Test Plan:
1. Main build passes, FSC passes
[canvas-builds-refspec=81cc926fe89d6efd9f604d0c573630318fb4eb86]
Change-Id: Ie21d242aabe9826353f05149abb304a85223cebc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263149
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-628
Test Plan:
1. Build summary report works for sub-jobs and normal stages
2. Sub-jobs run cleanupFn
[canvas-builds-refspec=82ccf7ae185871e45c658f225cc664ef05cda74d]
Change-Id: I9d4476b75e6488bcc388a89e9f7cf87150d0ae6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262951
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-621
Make the stage configuration aware of its node requirements for future EKS support and allowing nospot to be removed.
Test Plan:
1. RSpec / Selenium build works
2. Contract Tests works
3. JS works
4. FSC works
5. Vendored Gems works
[canvas-builds-refspec=480f3570a6a6ebee5cedaecbb3a089476a008215]
Change-Id: I6cb63df20abe8c1f913db4f1b21bb01bd4d30312
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262898
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-624
As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.
Test Plan:
1. FSC behaves correctly
Change-Id: Id5407fc398a78fab3f606a8e176cf1a620a8331f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262856
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-624
As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.
Test Plan:
1. All stages execute correctly
[canvas-builds-refspec=07f1a75ee23f8f6a9a1bbe062316373b761e7a30]
Change-Id: Ica11b78ebead5c9239dea112677a077654c7090d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262814
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
add the node number to the dir of archived
log files to help find the correct log.
flag = none
Test Plan:
-jenkins passes, fsc has logs in sub dirs with node num
Change-Id: Ifd79d38239fc683c0d5afda54689298489c1ebd1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258893
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Add timeout to all builds and set the proper values for the existing
timeouts
refs DE-307
flag = none
test-plan:
- Force the build to timeout and check that jenkins fails the build
Change-Id: I2e3ae36f828da1bc047bf5a2cfbc79c167b84d5f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256171
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
flag = none
Test Plan:
- Jenkins build passes with browser 87
- docker locally builds and runs selenium tests on chrome 87
- local install of canvas runs seleniums on chrome 87
Change-Id: Id1600e002c202a4dd90405d9b68a3ff092cbd764
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/255045
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
general rspec tests were running contract tests as well as the
contract tests build. contracts-generate-api.sh was duplicating
tests run in rspec.groovy.
test plan:
- build passes
- contract tests build runs contract tests as expected
- rspec build does NOT run contract tests
Change-Id: Ia3cacbf0ac099a52e016527800390d6dd17eb057
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254748
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
refs DE-351
flag = none
test-plan:
- Build passes
- Test FSC is working
Change-Id: Id3844dd12600e5d3a80d267bf67916c4d43d2548
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
These changes need to be accompanied by changes
to Jenkins config:
- main-canvas-postgres-12-ruby-2.6 and
main-canvas-postgres-12-ruby-2.6-postmerge, SCM
refspec changed to master
- sub-builds scm changed from canvas-lms-refspec
to JENKINSFILE_REFSPEC defaulting to master
- add Jenkins parameter JENKINSFILE_REFSPEC to sub-builds
- add new Jenkins build for main Jenkinsfile changes
- main build don't trigger on Jenkinsfile changes using
forbidden path.
Testing once merged:
- verify builds without Jenkinfile changes pull the master
version of Jenkinsfile
- verify sub-builds as well
- verify gerrit with Jenkinsfile changes triggers new build
- verify sub-builds get Jenkinsfiles from gerrit not master
- test a combination of Jenkinsfiles, not just the main one
- verify main build is not triggered
- run a plugin build without canvas-lms-refspec
- run a plugin build with canvas-lms-refspec
Change-Id: Ic6be8b446f80d57f7b0f25c77e6cffb959e03236
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251634
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
refs DE-349
[canvas-builds-refspec=f04c7902ccabadb61cee13fa93f09b1184925b84]
Test Plan:
1. Build pulls master when flag does not exist
2. Build pulls specified commit when flag exists
3. Build pulls master for post-merge when flag exists
Change-Id: I365efd3a305d08f7a3df76fd44799756104b4c1f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251635
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-23
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-23-test]
flag = none
Test Plan:
1. Jenkins pre-merge build with a new migration re-builds migrations
2. Jenkins pre-merge build with migrations cached does not re-build
3. Jenkins post-merge build with migrations cached re-builds migrations
4. skip-cache directive does not use the cached migration
5. Jenkins non-main builds works as expected
6. Jenkins FSC works as expected
Change-Id: I3221400a15220884740ad5136de7185a88ae2b39
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241013
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-295
Test Plan
1. Ensure that aborted build doesn’t report to slack
2. Ensure that timed out build reports to slack as warning
3. Ensure that successful build reports to slack as success
4. Ensure that failed build reports to slack as failure
Change-Id: I42117d95e4bf9808fcbed8b1f6def87815f06f66
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248838
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-297
Test Plan:
1. FSC works for plugin build
2. FSC includes all tests for incomplete / last partition
3. FSC properly balances tests across partitions
Change-Id: Ie2aa7bfbf37737b00184a4d1ce6c91fbd4a2ed4e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248840
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
refs DE-291
Test Plan:
1. Ensure that node times are approximately equal.
2. Ensure that FSC refuses to use over 10 nodes.
Change-Id: I08352fa231bb4f4acb7e64c1a3974895e495f26b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248756
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Brian Watson