Commit Graph

21 Commits

Author SHA1 Message Date
James Butters 5298bf7b9a add selenium version to jenkinsfile.selenium.fsc
flag = none

Change-Id: Iefd519cd2eedf9ada5a7756ded47b5040df4d713
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241546
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-06-30 17:17:05 +00:00
Rex Fleischer c5c3820042 use shared library for configuration
fixes: CCI-389

Test-Plan:
we need to run all the builds that we can for this:
- the current build needs to pass
- main-for-plugins needs to be ran
- master-bouncer
- dive?
- translations

Change-Id: Iae7f9dad7668b90653a0b9a8edc20ec69fa109c3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/240345
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ryan Norton <rnorton@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
2020-06-18 18:36:49 +00:00
Rex Fleischer 03bcc04e8c update fsc to go into the plugin directory to do tests
[skip-docker-build]

fixes: CCI-402
flag = none

Test-Plan:
execute the plugin build with this PS

Change-Id: Id8c85ef80671a5cd055e2aafd15cfb98d8251045
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/239678
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ryan Norton <rnorton@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
2020-06-17 14:58:09 +00:00
Rex Fleischer cf86e6d5ec change from using WORKSPACE to current directory for builds
while migrating to a build repo, we need to be able
to run these build scripts within the dir(){} on the checked
out root of this project. that will allow us to run these
scripts.

refs: CCI-393
flag = none

Test-Plan:
this build run and we test it with force-failure.
execute this with the canvas-builds while referencing this
PS to ensure that it will fix the WORKSPACE problem.

Change-Id: Idfe656771e2e4208e3b0980c942186a516f3bac1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/237905
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
2020-05-26 18:33:29 +00:00
James Butters 335f6c8c46 spec: more parallel of docker, better even
using the option project-name with docker-compose allows
us to have all services separate for each group of tests.

test plan:
Jenkins passes all build
logs are copied out of containers for archiving/test reports
run with force-failure flag


flag = none
closes: CCI-341

Change-Id: I99820f64023dde17a1b75e76bcec2dfe84529e91
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/236029
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-05-11 15:07:18 +00:00
Derek Bender 0f40402dfb Add timestamps to all builds
Closes: CCI-368

flag = none

Test Plan:
- All builds now have timestamps in the logs

Change-Id: I76f6b329c1b7f04511c6f63643964d6ccc8aa350
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/236719
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
2020-05-08 22:20:37 +00:00
Rex Fleischer 0bae9c44df add force failure for fsc
[skip-docker-build]

fixes: CCI-350
flag = none

Test-Plan:
add force failure to commit message and ensure fsc
actually gets forced to fail.

Change-Id: I3825dd389c9643ed8abb911cb0cd04b7101c6f9e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235673
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
2020-05-07 20:18:44 +00:00
James Butters c864cab227 spec: add postgres_password to fsc build
We missed adding the postgres password to fsc build.
This adds that env var and propagates the results.

FSC build working with changes:
https://jenkins.inst-ci.net/job/Canvas/job/test-suites/job/flakey-spec-catcher/4785/

flag = none
closes: CCI-355

Change-Id: Ia9eea1ed9c42f1946b9c840472d1e51383987e7f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235737
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-04-30 16:10:39 +00:00
Derek Bender 7e452e280e setup postgres extensions on all test dbs
Closes: CCI-345

Test Plan:
- set your COMPOSE_FILE to docker-compose.new-jenkins.yml
- remove any existing database volumes via
  `docker rm canvas-lms_pg_data` or `docker-compose down -v`
  which will remove all volumes in the project.

- update the image with `docker-compose build postgres`
- start the container

    $ docker-compose up --detach postgres

- create a second database:

    $ docker-compose exec -T postgres createdb -U postgres \
      -T template1 canvas_test1

- check both databases:

    $ seq 0 1 | parallel "docker-compose exec -T postgres psql -x \
      -U postgres \
      -d canvas_test{} \
      -c \"SELECT * FROM pg_available_extensions \
        WHERE name = 'pg_collkey';\"

- Expect two records to be present:

    -[ RECORD 1 ]-----+-------------------------------
    name              | pg_collkey
    default_version   | 0.5.1
    installed_version | 0.5.1
    comment           | ICU collation function wrapper

    -[ RECORD 1 ]-----+-------------------------------
    name              | pg_collkey
    default_version   | 0.5.1
    installed_version | 0.5.1
    comment           | ICU collation function wrapper

Change-Id: If1417df3edab788372927b16632e146069a6829e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235310
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
2020-04-27 18:25:57 +00:00
Rex Fleischer 06b0f6441f configure fsc to ignore gem plugin specs
fixes: CCI-306
flag = none

Test-Plan:
run the build with a test that should not be ran and
ensure it gets ignored.

PS4 showed the above

Change-Id: I55434d1e77c119e06e20b2a0cb2aca31fa94f498
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/234086
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
2020-04-15 17:16:23 +00:00
Brian Watson 55cc7e778e Re-enable FSC while not failing builds due to timeouts
flag = none

Test-plan: - PS4: Times out and propagates a success from FSC
    FSC build 2038
    Canvas build 25027
  - PS5: Fails a spec and propagates a failure
  - PS6: Passes
Change-Id: Ibc0313088e5f2128d143f6aaeb4712c6ba2bfedf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/231005
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Jacob Powell <spowell@instructure.com>
2020-03-24 15:09:44 +00:00
Derek Bender 68d03c9e94 Add verbose flags to builds
Closes: CCI-248
flag = none

Test Plan: builds pass

Change-Id: Ia0421d4036eb1655a10497363e0327779c69746b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/229446
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
2020-03-18 13:41:00 +00:00
Brian Watson 453f88d436 Add slack output to fsc
Test-plan:
 - look at previous PS and their corresponding FSC jobs
   and verify the jobs pass/fail accordingly
 - go to #flakey_spec_catcher_noisy and click the links
   verify that the correspond to the right builds
   and that the slack message colors correspond correctly
   to pass / fail

Change-Id: I00678ea186196f4202f6f3aa3620e67869bef26c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/229240
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
2020-03-12 15:46:00 +00:00
Derek Bender 78123c4cb6 Multiple rubies and postgres on Jenkins
Closes: CCI-214
flag = none

Test plan:
- The following builds pass:
  - main-canvas
  - main-for-coverage
  - main-canvas-postgres-12
  - main-canvas-ruby-2.6
  - main-canvas-postgres-12-ruby-2.6

Change-Id: I253d0cc8f0a808d8453fd3cbe1cfc3400ec8b12a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/227304
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
2020-03-10 22:30:51 +00:00
James Butters b5aa09359a FSC propagate false and reduce timeout
FSC needs some attention and is currently blocking
some patchsets from merging. Set propagate to false
while we work on some issues. Also reduce the timeout
to be similar to other build times.

flag = none

Change-Id: I0e2677fe3dea294c96c9285edd6b6519d34ed560
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/228212
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
Tested-by: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-02-27 17:07:44 +00:00
James Butters 9b683a0b82 change container to web
we're trying to copy from flakey_spec_catcher container, should
be web container for output file.

flag = none

Test Plan
  Jenkins passes

Change-Id: Ic17a3d6287ccd073ddff317e4bdadcd83844128d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/228096
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
2020-02-26 23:13:00 +00:00
Rex Fleischer 63a71e7cc7 make flakey spec catcher build potentially distributed
fixes: CCI-236
flag = none

Test-Plan:
get it passing as is, then make a version of this
PS have a bunch of test changes to force the distributed
path to trigger.

- test with one node
-- PS 11
-- https://jenkins.inst-ci.net/job/Canvas/job/test-suites/job/flakey-spec-catcher/299/
- test with multiple nodes
-- PS 14
-- https://jenkins.inst-ci.net/job/Canvas/job/test-suites/job/flakey-spec-catcher/306/

and for testing with no spec changes, the latest PS will have no tests

Change-Id: I4c69ce5c4cb67448ed270ae9fb3255ce21356b11
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/227214
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
2020-02-20 21:02:26 +00:00
James Butters 8294c8059d add use-parent option for flakey_spec_catcher build
flag = none

Change-Id: Ice5002596684c5000d1b8190b75dd16e7cf04a6f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/227057
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-02-18 22:10:21 +00:00
James Butters 8c9e1c4c0f spec: decrease timeout for fsc running tests
Decrease the timeout to a more acceptable time. Don't run FSC
on change-merged events.

flag = none

Change-Id: I4e2a4ceae64462f886c194e9c79977027b90c7f7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/226771
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-02-14 18:34:29 +00:00
James Butters 552c5b504c spec: archive output and cleanup
Add stage to copy over output from fsc for debugging. Add
cleanup post steps.

flag = none

Change-Id: Ibe8795e114f23efea86488285fe788bc8347a8de
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/226416
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-02-12 22:15:03 +00:00
Brian Watson 5dbd07d68c implement flakey_spec_catcher build
To optimize re-runs, FSC uses the same settings for
RSpec::Core::Runner but just clears the RSpec example queue
between runs. The changes in spec/selenium/common.rb ensure
that we don't start up a new webserver and driver when there
is already one existing. This creates connection issues.

closes: CCI-225

Test-plan:
 > export COMPOSE_FILE='docker-compose.new-jenkins.yml:docker-compose.new-jenkins-flakey-spec-catcher.yml:docker-compose.new-jenkins-selenium.yml'
 > docker-compose up -d
 > docker-compose run --rm web flakey_spec_catcher
 - verify that no specs are re-run
 - Try making changes to other selenium and api specs, re-run
 > docker-compose run --rm web flakey_spec_catcher
 - and make sure that the results appear as expected

Change-Id: I005ca9dec5e75f72788a7db5a9fa640f69071f6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/218480
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-02-11 19:39:47 +00:00