refs FOO-1644
[fsc-propagate=false]
- create engines directory
- put one empty-ish engine in there
- add general engines dependency to
gemfile construction
- update build pipeline to run tests all engines
just like gems
- add engines directory to jenkins-cache build
- get rspec running successfully with a dummy app
inside the audit engine
- stop failing with no message if environment
unloadable in css config
- generally set pattern in build artifacts and in
code organization for pulling out an engine
Change-Id: I8650f5e0aa7d72556d2f02156886d2b503a111fb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258988
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Add timeout to all builds and set the proper values for the existing
timeouts
refs DE-307
flag = none
test-plan:
- Force the build to timeout and check that jenkins fails the build
Change-Id: I2e3ae36f828da1bc047bf5a2cfbc79c167b84d5f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256171
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
we want to always checkout master code for testing on jenkins, unless
there are changes to our build scripts then pull the gerrit refspec.
flag = none
Test Plan:
- Normal Jenkins build passes.
- changes to docker-compose.new-jeknins* causes sub builds
to checkout code from gerrit refspec.
- changes to any file in build/new-jenkins/ causes sub
builds to checkout code from gerrit refspec.
- Normal build with no changes to Jenkinsfile*, build/new-jenkins/,
or docker-compose.new-jenkins* sub builds checkout master.
Change-Id: If1224a44b52b00f0f056b9552cd562809fc7eaa8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252999
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs DE-351
flag = none
test-plan:
- Build passes
- Test FSC is working
Change-Id: Id3844dd12600e5d3a80d267bf67916c4d43d2548
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
These changes need to be accompanied by changes
to Jenkins config:
- main-canvas-postgres-12-ruby-2.6 and
main-canvas-postgres-12-ruby-2.6-postmerge, SCM
refspec changed to master
- sub-builds scm changed from canvas-lms-refspec
to JENKINSFILE_REFSPEC defaulting to master
- add Jenkins parameter JENKINSFILE_REFSPEC to sub-builds
- add new Jenkins build for main Jenkinsfile changes
- main build don't trigger on Jenkinsfile changes using
forbidden path.
Testing once merged:
- verify builds without Jenkinfile changes pull the master
version of Jenkinsfile
- verify sub-builds as well
- verify gerrit with Jenkinsfile changes triggers new build
- verify sub-builds get Jenkinsfiles from gerrit not master
- test a combination of Jenkinsfiles, not just the main one
- verify main build is not triggered
- run a plugin build without canvas-lms-refspec
- run a plugin build with canvas-lms-refspec
Change-Id: Ic6be8b446f80d57f7b0f25c77e6cffb959e03236
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251634
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
refs DE-349
[canvas-builds-refspec=f04c7902ccabadb61cee13fa93f09b1184925b84]
Test Plan:
1. Build pulls master when flag does not exist
2. Build pulls specified commit when flag exists
3. Build pulls master for post-merge when flag exists
Change-Id: I365efd3a305d08f7a3df76fd44799756104b4c1f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251635
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-23
[change-merged]
[build-registry-path=jenkins/canvas-lms/de-23-test]
flag = none
Test Plan:
1. Jenkins pre-merge build with a new migration re-builds migrations
2. Jenkins pre-merge build with migrations cached does not re-build
3. Jenkins post-merge build with migrations cached re-builds migrations
4. skip-cache directive does not use the cached migration
5. Jenkins non-main builds works as expected
6. Jenkins FSC works as expected
Change-Id: I3221400a15220884740ad5136de7185a88ae2b39
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241013
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
fixes: CCI-389
Test-Plan:
we need to run all the builds that we can for this:
- the current build needs to pass
- main-for-plugins needs to be ran
- master-bouncer
- dive?
- translations
Change-Id: Iae7f9dad7668b90653a0b9a8edc20ec69fa109c3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/240345
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ryan Norton <rnorton@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
Closes: CCI-368
flag = none
Test Plan:
- All builds now have timestamps in the logs
Change-Id: I76f6b329c1b7f04511c6f63643964d6ccc8aa350
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/236719
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
This patchset removes the much duplicated database.yml and instead
defaults to a DATABASE_URL scheme. There is however still present a
docker-compose/config/new-jenkins/database.yml file to temporarily allow
our internal portal service to still function until such time we can
reconfigure it.
To set up custom connection options all that is needed is setting
DATABASE_URL. See examples of this in docker-compose.yml and
docker-compose.new-jenkins.yml.
Test Plan
=========
Image:
- pulling canvas-lms:master image boots without database errors
Repo:
- given default dev environment where
COMPOSE_FILE=docker-compose.yml:docker-compose.override.yml
- build the web and postgres images: docker-compose build web postgres
(feel free to comment out `yarn install` and `compile_assets` in the
Dockerfile since we won't be needing them for this test and this
speeds things up)
- set a custom DATABASE_URL to change the database name and echo it to
see it gets into the image correctly:
$ docker run -it -e \
DATABASE_URL=postgres://postgres:sekret@postgres/canvas_custom_env \
$(docker-compose ps -q web) bash -c "echo \$DATABASE_URL"
=> postgres://postgres:sekret@postgres/canvas_custom_env
$ docker-compose run --rm -e \
DATABASE_URL=postgres://postgres:sekret@postgres/canvas_custom_env \
web bash -c "echo \$DATABASE_URL"
=> postgres://postgres:sekret@postgres/canvas_custom_env
- Test that setting a custom database name via DATABASE_URL can create
the database
$ docker-compose run --rm -e \
DATABASE_URL=postgres://postgres:sekret@postgres/canvas_custom_env \
web bash -c "bin/rails db:create"
=> Created database 'canvas_custom_env'
- This patchset boots on portal
Change-Id: Ic6f3819550df94b6c98b17ef05ac2029a578cfe4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/234213
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
Closes: CCI-345
Test Plan:
- set your COMPOSE_FILE to docker-compose.new-jenkins.yml
- remove any existing database volumes via
`docker rm canvas-lms_pg_data` or `docker-compose down -v`
which will remove all volumes in the project.
- update the image with `docker-compose build postgres`
- start the container
$ docker-compose up --detach postgres
- create a second database:
$ docker-compose exec -T postgres createdb -U postgres \
-T template1 canvas_test1
- check both databases:
$ seq 0 1 | parallel "docker-compose exec -T postgres psql -x \
-U postgres \
-d canvas_test{} \
-c \"SELECT * FROM pg_available_extensions \
WHERE name = 'pg_collkey';\"
- Expect two records to be present:
-[ RECORD 1 ]-----+-------------------------------
name | pg_collkey
default_version | 0.5.1
installed_version | 0.5.1
comment | ICU collation function wrapper
-[ RECORD 1 ]-----+-------------------------------
name | pg_collkey
default_version | 0.5.1
installed_version | 0.5.1
comment | ICU collation function wrapper
Change-Id: If1417df3edab788372927b16632e146069a6829e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235310
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
QA-Review: Jacob Powell <spowell@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Closes: CCI-344
Test Plan: No mention of spring in any stage that uses
build/new-jenkins/docker-compose-create-migrate-database.sh
Change-Id: Iafe5bb635bf936f83fdad1170398bb8852ebb804
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/235312
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Closes: CCI-339
flag=none
test plan: spring is not referenced in builds
Change-Id: I6c3fb84f7e5d84d6507698b77495b8d376867fb3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/234826
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Filters keys ending in _KEY or _SECRET
Closes: CCI-202
flag = none
Test Plan:
- keys like GERGICH_KEY are no longer printed in the builds
Change-Id: I3c6b7cc882ba36b291dc44ac5fe9c958aa24cd60
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/224898
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
fixes: CCI-151
flag = none
Test-Plan:
make sure it runs through the pipeline correctly
Change-Id: Ib1b76010b8122dcb76d9f257a1f2d674f6afc2c1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/220247
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Robert Lamb <rlamb@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
fixes: CCI-148
flag = none
Test-Plan:
look through the logs on each build and ensure the pre-clean runs
Change-Id: I1e10ab43deeb4dbcaa7b9c4f08ea3737675ebdf4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/220000
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: Robert Lamb <rlamb@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
also added a commit flag checker and moved the credential
methods to another location to share in linters build.
fixes: CCI-80
linters build changes:
- added build param: GERRIT_CHANGE_COMMIT_MESSAGE
- added build param: GERRIT_HOST
- added build param: GERGICH_PUBLISH
Test-Plan:
made some changes and saw gergich send -1 those changes.
Change-Id: I8ff32e1e890f495d4afd0f9791ff183dfeabf750
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/217416
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: S. Jacob Powell <spowell@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Derek Bender <djbender@instructure.com>
QA-Review: Rex Fleischer <rfleischer@instructure.com>
Product-Review: Rex Fleischer <rfleischer@instructure.com>
so we don't get rate-limited from docker hub
closes #CORE-3344
Change-Id: I5c8f0e1716a84966dc9b0f0f8595a474649f89cb
Reviewed-on: https://gerrit.instructure.com/211250
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Jenkins
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Closes CORE-3304
Test plan:
Jenkins passes
Verify builds run on canvas_docker nodes
Change-Id: I789531b8962d7ca53ff938951366a173ed69f2ca
Reviewed-on: https://gerrit.instructure.com/208506
Tested-by: Jenkins
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Robert Lamb <rlamb@instructure.com>
Product-Review: Robert Lamb <rlamb@instructure.com>
add cleanup for spec_results folder on nodes.
add jasmine-pretty-html-reporter to get html reports
for i18n tests.
refs: CORE-3254
Change-Id: Ifa1724e368c5c91dc69e854ae98a1a13abe33aa0
Reviewed-on: https://gerrit.instructure.com/207203
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
For New Jenkins builds, don't fallback to pulling the image from
master if the patchset image isn't available. We don't want the
tests to pass against the wrong image.
Use PATCHSET_TAG instead of re-creating the starlord url.
Change-Id: Id77e4110f597308064922c1bb37d03b520a4d6b9
Reviewed-on: https://gerrit.instructure.com/206742
Tested-by: Jenkins
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
most builds do not need selenium container so move the selenium
config into it's own yml and only add to env variable when
needed.
Removed docker-compose-build.sh, add docker-compose build code
into the build scripts.
closes: CORE-3253
Change-Id: I5698019c69dd8940624589928b8a7d4b65f940a5
Reviewed-on: https://gerrit.instructure.com/206247
Tested-by: Jenkins
Reviewed-by: Robert Lamb <rlamb@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Aaron Ogata