RspecQ can now be used to run tests using the
'use-rspecq' commit message feature flag
test plan:
- build runs as normal without use-rspecq enabled
- enabling use-rspecq commit flag runs tests using rspecq,
no tests run via old rspec/slenium groups
- test numbers match between differing builds
- build artifacts are created with or without feature flag
- build summary report works with or without feature
flag
Change-Id: Ibaf32177a8ec28a89278eaa4277bd7a752cbfc58
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272130
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-644
[canvas-builds-refspec=c30dc5a5768e38eb7ef483a6b8b5e7c30af8a25d]
Test Plan
1. RSpec / Selenium use correct number of nodes and run correct number of tests
2. RSpec / Selenium retry according to RERUNS_RETRY value
3. Build Summary Report generates correctly
Change-Id: I886e41a8712bda5a98f58eee0ba42d034a08db25
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265014
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-444
This extra parameter isn’t used anywhere, so remove it.
Change-Id: I487e7a6d73fdd218f87263fdf1a8970057ae3ef8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264900
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-467
it is no longer possible to spin up multiple docker containers
ona single rspec or selenium agent when running tests. parallelization
is now exclusively done through multple rspec processes on
a single container.
test plan:
- setting docker_processes ENV variable has no effect
- settting rspec_processes still runs multiple test threads
- tests continue to run as expected given that docker_processes has
been set to 1 for multiple months.
Change-Id: I604d953a71ad220373b6a71d558f46bff29f820e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263701
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
adding variables to use rails 6 in new test build.
Test Plan:
- Jenkins build pass
- rails 5 is installed and used
- Run Main-from-plugin
- rails 5 is installed and used
- Run new rails6 build
- rails 6 is installed and used
- Run new rails6 plugin build
- rails 6 is installed and used
Change-Id: Ib4441f2eb6fb8946089c17bf81ec16bcb7b42020
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250840
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs DE-23
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-23-test]
flag = none
Test Plan:
1. Jenkins pre-merge build with a new migration re-builds migrations
2. Jenkins pre-merge build with migrations cached does not re-build
3. Jenkins post-merge build with migrations cached re-builds migrations
4. skip-cache directive does not use the cached migration
5. Jenkins non-main builds works as expected
6. Jenkins FSC works as expected
Change-Id: I3221400a15220884740ad5136de7185a88ae2b39
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241013
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
closes: DE-119
flag = none
Test Plan:
-Regular jenkins builds pass
-New build Canvas/test-suites/rspec-parallel-log
-passes and pushes gerrit with new log
-fails and does not run create gerrit stage
-once one step fails, entire build is failed immediately
Change-Id: I965676e26927ec5b539532cdf87191cc52db9151
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242157
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
combine docker-compose.new-jenkins.yml and
docker-compose.new-jenkins.multple-processes.yml into one since
they are very similar.
flag = none
test plan
-All Jenkins builds still pass
Change-Id: I63737562aa054cd471fbd165227b928f93731371
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241393
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
This will allow use to rely upon builtin "tini" functionality
without needing to manually invoke it like before. This should
ensure that stopping containers is faster across the board.
Change-Id: I101770a6ddcf5cdc216b612bde83156d40198127
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/240309
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
These images are currently pulled from the cache or built in docker-compose-pull.sh; and docker-compose-build-up.sh is rebuilding them. Fix this so that the cached image is used for our build files.
refs DE-3
Test Plan:
1. Ensure Jenkins passes
flag = none
Change-Id: Id8b4954ee452d2534b898f9d782b67b9730f542c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/239644
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
using the option project-name with docker-compose allows
us to have all services separate for each group of tests.
test plan:
Jenkins passes all build
logs are copied out of containers for archiving/test reports
run with force-failure flag
flag = none
closes: CCI-341
Change-Id: I99820f64023dde17a1b75e76bcec2dfe84529e91
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/236029
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
This patchset removes the much duplicated database.yml and instead
defaults to a DATABASE_URL scheme. There is however still present a
docker-compose/config/new-jenkins/database.yml file to temporarily allow
our internal portal service to still function until such time we can
reconfigure it.
To set up custom connection options all that is needed is setting
DATABASE_URL. See examples of this in docker-compose.yml and
docker-compose.new-jenkins.yml.
Test Plan
=========
Image:
- pulling canvas-lms:master image boots without database errors
Repo:
- given default dev environment where
COMPOSE_FILE=docker-compose.yml:docker-compose.override.yml
- build the web and postgres images: docker-compose build web postgres
(feel free to comment out `yarn install` and `compile_assets` in the
Dockerfile since we won't be needing them for this test and this
speeds things up)
- set a custom DATABASE_URL to change the database name and echo it to
see it gets into the image correctly:
$ docker run -it -e \
DATABASE_URL=postgres://postgres:sekret@postgres/canvas_custom_env \
$(docker-compose ps -q web) bash -c "echo \$DATABASE_URL"
=> postgres://postgres:sekret@postgres/canvas_custom_env
$ docker-compose run --rm -e \
DATABASE_URL=postgres://postgres:sekret@postgres/canvas_custom_env \
web bash -c "echo \$DATABASE_URL"
=> postgres://postgres:sekret@postgres/canvas_custom_env
- Test that setting a custom database name via DATABASE_URL can create
the database
$ docker-compose run --rm -e \
DATABASE_URL=postgres://postgres:sekret@postgres/canvas_custom_env \
web bash -c "bin/rails db:create"
=> Created database 'canvas_custom_env'
- This patchset boots on portal
Change-Id: Ic6f3819550df94b6c98b17ef05ac2029a578cfe4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/234213
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
Closes: CCI-345
Test Plan:
- set your COMPOSE_FILE to docker-compose.new-jenkins.yml
- remove any existing database volumes via
`docker rm canvas-lms_pg_data` or `docker-compose down -v`
which will remove all volumes in the project.
- update the image with `docker-compose build postgres`
- start the container
$ docker-compose up --detach postgres
- create a second database:
$ docker-compose exec -T postgres createdb -U postgres \
-T template1 canvas_test1
- check both databases:
$ seq 0 1 | parallel "docker-compose exec -T postgres psql -x \
-U postgres \
-d canvas_test{} \
-c \"SELECT * FROM pg_available_extensions \
WHERE name = 'pg_collkey';\"
- Expect two records to be present:
-[ RECORD 1 ]-----+-------------------------------
name | pg_collkey
default_version | 0.5.1
installed_version | 0.5.1
comment | ICU collation function wrapper
-[ RECORD 1 ]-----+-------------------------------
name | pg_collkey
default_version | 0.5.1
installed_version | 0.5.1
comment | ICU collation function wrapper
Change-Id: If1417df3edab788372927b16632e146069a6829e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235310
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Closes: CCI-344
Test Plan: No mention of spring in any stage that uses
build/new-jenkins/docker-compose-create-migrate-database.sh
Change-Id: Iafe5bb635bf936f83fdad1170398bb8852ebb804
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235312
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Using the parallel_tests gem allows us to parallelize more
and a bit simpler. Removes headless stuff now that we don't
use it anymore, comments out video recording code since it
doesn't work, removes all knapsack code.
flag = none
Test Plan:
Jenkins rspec and selenium builds pass
rspec and selenium builds copy results to html publisher
rspec and selenium have nodes split into processes
Change-Id: Ibd00eba77f8193be5eadd41aef1e0617c9ae470c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/227677
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Ryan Taylor <rtaylor@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Closes: CCI-261
The mdillon repo was an unofficial postgis repo, there is now and
official postgis/postgis repo!
https://registry.hub.docker.com/r/postgis/postgis/https://github.com/postgis/docker-postgis
Test Plan:
- First, take notice if you currently have pg_collkey installed:
$ docker-compose run --rm postgres psql -x \
-h postgres
-U postgres
-d canvas
-c "SELECT * FROM pg_available_extensions WHERE \
pg_available_extensions.name = 'pg_collkey';"
If `installed_version` is blank, then the extension is not isntalled
into the `canvas` database.
- take down any running environment:
$ docker-compose stop && docker-compose down
- copy (or update) new settings from docker-compose.override.yml:
# manual diff
$ diff config/docker-compose.override.yml \
docker-compose.override.yml
# overwrite any existing config
$ cp config/docker-compose.override.yml .
- copy (or update) new settings from database.yml:
# manual diff
$ diff docker-compose/config/database.yml config/database.yml
# overwrite any existing config
$ cp docker-compose/config/database.yml config/
- rebuild and startup the postgres image:
$ docker-compose up --build postgres
- the postgres logs do not mention
ERROR: could not access file "$libdir/collkey_icu.so":
No such file or directory
- install the extension if it wasn't previously present:
$ docker-compose run --rm postgres psql -x \
-h postgres \
-U postgres \
-d canvas \
-c "CREATE EXTENSION IF NOT EXISTS pg_trgm SCHEMA public;
CREATE EXTENSION IF NOT EXISTS postgis SCHEMA public;
CREATE EXTENSION IF NOT EXISTS pg_collkey SCHEMA public;"
$ docker-compose run --rm postgres psql -x \
-h postgres \
-U postgres \
-d canvas_test \
-c "CREATE EXTENSION IF NOT EXISTS pg_trgm SCHEMA public; \
CREATE EXTENSION IF NOT EXISTS postgis SCHEMA public; \
CREATE EXTENSION IF NOT EXISTS pg_collkey SCHEMA public;"
- pg_collkey is now present:
$ docker-compose run --rm postgres psql -x \
-h postgres
-U postgres
-d canvas
-c "SELECT * FROM pg_available_extensions WHERE \
pg_available_extensions.name = 'pg_collkey';"
$ docker-compose run --rm postgres psql -x \
-h postgres
-U postgres
-d canvas_test
-c "SELECT * FROM pg_available_extensions WHERE \
pg_available_extensions.name = 'pg_collkey';"
- Optionally, Build the postgres container with fresh volumes:
$ COMPOSE_PROJECT_NAME=<a-unique-name> docker-compose --build \
up postgres
- The postgres logs show no errors related to collkey_icu.so
(like above)
Change-Id: I4f0e027cd908fee937b6204cd3a20e5ecf998021
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/228190
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
test plan:
Passes jenkins including on merge
No longer builds the database images, it pulls them instead
fixes: CCI-233
Change-Id: I15d96fc901115aa0d1718c1b5794898524b8eb59
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/226597
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Jacob Powell <spowell@instructure.com>
Tested-by: James Butters <jbutters@instructure.com>
Tested-by: Rex Fleischer <rfleischer@instructure.com>
fixes: CCI-170
flag = none
Test-Plan:
manually main-for-coverage with this specref and ensure
coverage is uploaded
Change-Id: Ifc67a65157e8ad5853b01664fe81e177f8bba0d6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/226086
QA-Review: Rex Fleischer <rfleischer@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
To optimize re-runs, FSC uses the same settings for
RSpec::Core::Runner but just clears the RSpec example queue
between runs. The changes in spec/selenium/common.rb ensure
that we don't start up a new webserver and driver when there
is already one existing. This creates connection issues.
closes: CCI-225
Test-plan:
> export COMPOSE_FILE='docker-compose.new-jenkins.yml:docker-compose.new-jenkins-flakey-spec-catcher.yml:docker-compose.new-jenkins-selenium.yml'
> docker-compose up -d
> docker-compose run --rm web flakey_spec_catcher
- verify that no specs are re-run
- Try making changes to other selenium and api specs, re-run
> docker-compose run --rm web flakey_spec_catcher
- and make sure that the results appear as expected
Change-Id: I005ca9dec5e75f72788a7db5a9fa640f69071f6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/218480
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Closes: CCI-7
flag=none
Test Plan:
- Main build passes normally
- Trigger a build manually for `selenium-performance-chrome` with this
patchset's refs/changes/18/219818/<PATCHSET_NUMBER>
- The build should run normally without configuration type failures
Change-Id: Icc67d1274783b47166ea10f09061cc76727fbc62
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/219818
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: Robert Lamb <rlamb@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Overview:
This patchset is intended to save time by saving and loading migrated
containers and reusing the image.
It checks to see if all three database images were loaded (the cached
ones) and if so, then it uses those instead of migrating from scratch.
If one of them is not loaded, it performs the regular process.
The storing of the migrated images will only happen on merge.
It does run migrate on the cached images, but it correctly skips
anything that was already done.
This pattern will be repeated in each sub build, which should save a lot
of time not having to redo the migrations.
Test plan:
View the build and confirm that it passes when loading cached migrations
Should be about a minute or more faster
Fixes: CCI-30
Change-Id: Iba878bb646fd976deab5d624e8960cac63d09677
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/213268
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: S. Jacob Powell <spowell@instructure.com>
Product-Review: S. Jacob Powell <spowell@instructure.com>
this avoids a bunch of warnings in the un-knapsacked builds
Change-Id: I5caf6bd94d5d7b3350a492b739f6d712c7e34829
Reviewed-on: https://gerrit.instructure.com/211667
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Tested-by: Jenkins
Change-Id: I5728c9cd0c36c42d79fd0da0f0e9b2849df54c7f
Reviewed-on: https://gerrit.instructure.com/210582
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
so we don't get rate-limited from docker hub
closes #CORE-3344
Change-Id: I5c8f0e1716a84966dc9b0f0f8595a474649f89cb
Reviewed-on: https://gerrit.instructure.com/211250
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Jenkins
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Change-Id: I30c0c0006de8779dea1743d6c626022756f05d5a
Reviewed-on: https://gerrit.instructure.com/210564
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Change-Id: Id4a31fe3d3c8784b65730d7c62d3bd1ef68767ed
Reviewed-on: https://gerrit.instructure.com/210756
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Jenkins
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
For New Jenkins builds, don't fallback to pulling the image from
master if the patchset image isn't available. We don't want the
tests to pass against the wrong image.
Use PATCHSET_TAG instead of re-creating the starlord url.
Change-Id: Id77e4110f597308064922c1bb37d03b520a4d6b9
Reviewed-on: https://gerrit.instructure.com/206742
Tested-by: Jenkins
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
most builds do not need selenium container so move the selenium
config into it's own yml and only add to env variable when
needed.
Removed docker-compose-build.sh, add docker-compose build code
into the build scripts.
closes: CORE-3253
Change-Id: I5698019c69dd8940624589928b8a7d4b65f940a5
Reviewed-on: https://gerrit.instructure.com/206247
Tested-by: Jenkins
Reviewed-by: Robert Lamb <rlamb@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Jacob Burroughs