Commit Graph

27 Commits

Author SHA1 Message Date
Aaron Ogata 1e6312a404 move nodeLabel out of canvas-builds
refs DE-1359

Change-Id: I09cbcb708f9f8451a205b4fc470c27224bac49c1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/313499
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
2023-03-16 19:41:00 +00:00
Aaron Ogata 32e454884a remove remaining use cases of configuration.getString()
refs DE-1359

Change-Id: Ib0680f9e97ce833aa6fdae0f7e4ae950bbcc89a2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/313481
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2023-03-16 18:06:15 +00:00
Aaron Ogata 80a5f8fc5d replace checkoutRepo with checkoutFromGit
refs DE-1559

[canvas-builds-refspec=4e6cb119dedfbcbeea5045e2cc77754a568088cd]

Change-Id: Iff0c62d0ea6da2359b786a474d7ab735a16c2dde
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/313117
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
2023-03-13 21:17:18 +00:00
James Butters 2b7fc11650 remove code to delete Gemfile.lock
we no longer need to remove the Gemfile.lock, remove the code that
does this as part of setup.

Test Plan:
- Jenkins docker smoke passes
- running docker_dev_setup.sh locally passes

Change-Id: Ic8807497f0f7489028ae474f0774cc5f871cb0f9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/311951
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Build-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2023-02-24 18:00:53 +00:00
Aaron Ogata 983bc779d3 allow node label to be overridden for testing
refs DE-1258

[canvas-builds-refspec=2b6856c05a90d298d5c25fb613233df9bc22af12]

Change-Id: Ic529fc5afce5a06c94a7efc71f8da01a55142c69
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/295032
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2022-06-29 14:41:05 +00:00
James Butters 1432d3e72a Revert "Replace old config ymls with newer mutagen ymls"
This reverts commit d3fc54655d.

Reason for revert: broke M1, non-mutagen setup

Change-Id: Iddcfa0b086312b4ed22f400a61637ee248fd647d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/288011
Tested-by: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2022-03-28 17:38:37 +00:00
James Butters d3fc54655d Replace old config ymls with newer mutagen ymls
flag = none

Test Plan:
-Jenkins passes
-running docker_dev_setup works from fresh machine
-running docker_dev_update works

Change-Id: I7852189b035683d7cb538b2a04319dd9659e2752
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/287799
Reviewed-by: Bobby Buten <bobby.buten@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2022-03-25 15:38:49 +00:00
James Butters cc32af3ccd Revert "asset install in Dockerfile"
This reverts commit 655cecbd5d.

Reason for revert: broke dev environments

Change-Id: I97f47a8ef7ac369b3cb30e448a3cfa9997c03930
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267650
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2021-06-29 13:36:09 +00:00
James Butters 655cecbd5d asset install in Dockerfile
move asset compiling into the Dockerfile allowing for a simpler
docker dev setup.

flag = none
closes: DE-696,DE-365

Test Plan:
-mac normal setup
-mac mutagen setup
-linux normal setup
-linux mutagen setup

Change-Id: I7e178c1ca91ee1908c4af55e03ec0010f5abce8e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266332
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
2021-06-24 19:25:48 +00:00
Andrea Cirulli 3f057e9c0c fix jenkins files linter issues
Fix linter issues related to variable name, space after closing bracket.

refs DE-690
flag = none

test-plan:
- Build passes
- QA build with spec change executes fsc stage

Change-Id: I6f985fb3dfa8627703a27325b1eb25fae4585505
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265188
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2021-05-18 18:58:33 +00:00
Andrea Cirulli 67d6ccc360 fix jenkins file linter issues autofixable
refs DE-690
flag = none

test-plan:
- Build passes

Change-Id: Ie748a1e585ce8e3ec4978c7a5faa45dab524a905
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265184
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2021-05-18 16:48:15 +00:00
James Butters e38b3a49be add support for mutagen setup and update.
flag = none
closes: DE-529

Test Plan:
Test All On Linux and Mac
-docker_dev_setup without mutagen
  -test with dinghy not created, down, missing
-docker_dev_setup with mutagen
  -with docker desktop not running
  -with and without dory
-docker_dev_update with and without mutagen
  -with update-code option and without
-update_canvas for local is not broken
-following Next Steps, all works for mutagen

Change-Id: I7690dc2d919cf1b9d8af86200ec8439da9135d16
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262293
QA-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2021-04-27 19:46:36 +00:00
James Butters c2bf287a2b set DOCKER env var to true, following our standard
flag = none
refs: DE-529

Test Plan:
-Jenkins docker smoke build passes
-running canvas_update local does not use docker commands
-running docker_dev_setup and docker_dev_update still works
 and use docker commands.

Change-Id: I44c0cd535f6857bebf6a3290efae6800c62c7c77
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263235
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2021-04-20 17:11:45 +00:00
Kyle Rosenbaum 5b996428a1 fix /usr/src/app permissions in local docker dev smoke test
refs DE-594

test plan:
- run local docker dev smoke test build in Jenkins
- verify permissions of application directory
- modify jenkins agent workspace to include
application directory.
- rerun local docker dev smoke test build
- verify permissions of application directory

Change-Id: I86bd669af30d7cc0286cccd5f6771f91a0f4afe9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261717
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-04-15 21:51:46 +00:00
Aaron Ogata e18887bf26 move docker cleanup to protectedNode() handler
refs DE-624

As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.

Test Plan:
1. All stages execute correctly

[canvas-builds-refspec=07f1a75ee23f8f6a9a1bbe062316373b761e7a30]

Change-Id: Ica11b78ebead5c9239dea112677a077654c7090d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262814
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-04-13 18:02:41 +00:00
James Butters 16cea75a47 move to docker-compose exec
to save time and clear up logs, start web contain and
dependant services then run docker-compose exec instead
of docker-compose run --rm for each command. No more
waiting for container to spin up and stop, also clears
the logs of all those extra messages.

closes: DE-593
flag = none

Test Plan:
-run docker_dev_setup.sh, should be no noticable difference
-run docker_dev_update.sh, should be no noticable difference
-Jenkins local-docker-dev build passes
-canvas_update for local non-docker still works

Change-Id: Id1e72f8af948905385aed6769a8d793d962e2e1f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262258
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2021-04-13 15:43:53 +00:00
James Butters 2b45e82021 abstract canvas build steps into separate script
closes: DE-567
flag = none

Test Plan:
-Jenkins passes
-docker_dev_setup.sh still works
-docker_dev_update.sh still works
-rebase_canvas_and_plugins.sh still works

Change-Id: I044ba0d3d8539a1e87bb0fef67f3b82855f4ab2c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260842
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2021-03-17 20:19:41 +00:00
James Butters 3a1eb90728 remove common_build_steps
remove common_build_steps and move those functions into
script/common.sh where they can be reused in all scripts.
Fix the docker-smoke build to still work with the move.

flag = none

Test Plan:
-Jenkins passes
-Local-docker-dev build passes
-docker_dev_setup script still works
-docker_dev_update script still works
-rebase_canvas_and_plugins script still works


Change-Id: I0300b90d631701a36ede5bfa52fad7e2406bb209
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260384
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2021-03-11 15:15:39 +00:00
Aaron Ogata 3e92cc4498 move execute function under libraryScript
refs DE-442

[canvas-builds-refspec=eecea52ff75a74be10cd0d2c1434dc917bd726af]

Change-Id: I7412d590e1c08a47eb48b1fd82af9eaf0d5eb145
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256507
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-01-12 00:32:08 +00:00
Andrea Cirulli 4d672c2f6a add global timeout and reevaluate all timeouts
Add timeout to all builds and set the proper values for the existing
timeouts

refs DE-307
flag = none

test-plan:
- Force the build to timeout and check that jenkins fails the build

Change-Id: I2e3ae36f828da1bc047bf5a2cfbc79c167b84d5f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256171
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2021-01-11 21:24:03 +00:00
James Butters a3178a0a75 improve docker_dev_setup.sh
remove redundant compiling of assets. Add setup for selenium tests with
info on how to get that setup. Reduce number of times attempting to
run migrations.

flag = none
closes: DE-364

Test Plan:
- Run docker_dev_setup.sh on clean environment
- Run docker_dev_setup.sh on linux machine
- Jenkins docker build passes

Change-Id: I0b553a820d8e472f40f1d9a71d282dac79f67027
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252024
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
2020-12-09 14:53:04 +00:00
James Butters cc45c56cb3 always checkout master code
we want to always checkout master code for testing on jenkins, unless
there are changes to our build scripts then pull the gerrit refspec.

flag = none

Test Plan:
- Normal Jenkins build passes.
- changes to docker-compose.new-jeknins* causes sub builds
  to checkout code from gerrit refspec.
- changes to any file in build/new-jenkins/ causes sub
  builds to checkout code from gerrit refspec.
- Normal build with no changes to Jenkinsfile*, build/new-jenkins/,
  or docker-compose.new-jenkins* sub builds checkout master.

Change-Id: If1224a44b52b00f0f056b9552cd562809fc7eaa8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252999
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-11-18 21:48:19 +00:00
Andrea Cirulli ba4ad02c38 abstract checkout code in jenkinsfiles
refs DE-351
flag = none

test-plan:
- Build passes
- Test FSC is working

Change-Id: Id3844dd12600e5d3a80d267bf67916c4d43d2548
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2020-11-04 20:58:00 +00:00
James Butters 986c935bdf checkout Jenkinsfile from master
These changes need to be accompanied by changes
to Jenkins config:
  - main-canvas-postgres-12-ruby-2.6 and
  main-canvas-postgres-12-ruby-2.6-postmerge, SCM
  refspec changed to master
  - sub-builds scm changed from canvas-lms-refspec
  to JENKINSFILE_REFSPEC defaulting to master
  - add Jenkins parameter JENKINSFILE_REFSPEC to sub-builds
  - add new Jenkins build for main Jenkinsfile changes
  - main build don't trigger on Jenkinsfile changes using
    forbidden path.

Testing once merged:
- verify builds without Jenkinfile changes pull the master
  version of Jenkinsfile
  - verify sub-builds as well
- verify gerrit with Jenkinsfile changes triggers new build
  - verify sub-builds get Jenkinsfiles from gerrit not master
  - test a combination of Jenkinsfiles, not just the main one
  - verify main build is not triggered
- run a plugin build without canvas-lms-refspec
- run a plugin build with canvas-lms-refspec

Change-Id: Ic6be8b446f80d57f7b0f25c77e6cffb959e03236
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251634
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
2020-11-03 03:08:08 +00:00
Aaron Ogata 2dcc3df114 introduce canvas-builds-refspec commit flag
refs DE-349

[canvas-builds-refspec=f04c7902ccabadb61cee13fa93f09b1184925b84]

Test Plan:
1. Build pulls master when flag does not exist
2. Build pulls specified commit when flag exists
3. Build pulls master for post-merge when flag exists

Change-Id: I365efd3a305d08f7a3df76fd44799756104b4c1f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251635
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-02 19:33:54 +00:00
James Butters 80300923b1 add slack alert for failed docker-dev cron jobs
flag = none
closes: DE-294

Change-Id: Ib88185e9c8207098f215e4a0e3f442f9d273dbd7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249189
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-10-05 14:23:36 +00:00
James Butters 5deafbc281 add docker-smoke build
Adding a new build for local docker setup, add new Jenkinsfile
for that build that replicates the local setup for docker dev.

closes: DE-207
flag = none

Test Plan:
- Run new Jenkinsfile in proof-of-concept build
- Current builds all pass, no effect on them
- New build passes as sub-build of main build
- New build triggered with changes to relelvant files

Change-Id: I1a2562425a33caee6eed4c1e2899c364dd181711
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245728
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-09-14 23:12:35 +00:00