Commit Graph

81 Commits

Author SHA1 Message Date
Aaron Ogata 1e6312a404 move nodeLabel out of canvas-builds
refs DE-1359

Change-Id: I09cbcb708f9f8451a205b4fc470c27224bac49c1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/313499
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
2023-03-16 19:41:00 +00:00
Aaron Ogata a6aae57a48 convert remaining boolean flags
refs DE-1359

[canvas-builds-refspec=1f02879d6f1b3a3ca631b542539aa0b699e64146]

Change-Id: Id0b5e194f7e7d1ba55723fcbb5c0dcede028c5de
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/313490
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
2023-03-16 18:30:43 +00:00
James Butters b3ba447056 updgrade Jenkins Chrome version 107
closes: DE-1483

Test Plan:
- new image starlord.inscloudgate.net/jenkins/selenium-node-chrome:107.0
- Jenkins build uses chrome 107
- No failures in Jenkins

Change-Id: I31f453d34dcfcbc71c88b3bc379380bbd17ef133
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/306441
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Build-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Ryan Norton <rnorton@instructure.com>
2022-12-06 02:28:57 +00:00
Aaron Ogata 33c90aa925 stop loading webpack-builder image twice before running JS specs
refs DE-1345

saves ~90s when running JS stages

Change-Id: I3cdb97193e8b420e7eed3f63eecedf7a12a5d907
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/298579
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Alex Slaughter <aslaughter@instructure.com>
2022-08-17 13:45:40 +00:00
Aaron Ogata 983bc779d3 allow node label to be overridden for testing
refs DE-1258

[canvas-builds-refspec=2b6856c05a90d298d5c25fb613233df9bc22af12]

Change-Id: Ic529fc5afce5a06c94a7efc71f8da01a55142c69
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/295032
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2022-06-29 14:41:05 +00:00
Aaron Ogata 433bbef122 split packages into own node
refs DE-1215

Change-Id: I601bf51238c69e8eaa7e74a093ffcebec77e411e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/293024
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Ryan Norton <rnorton@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
2022-06-01 19:07:02 +00:00
Aaron Ogata b728c44fa3 record more build timing metrics
refs DE-1207

Change-Id: Ifeda184cddabf24debbb56848ebd22dc4e9e7eeb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/292966
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Alex Slaughter <aslaughter@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2022-06-01 03:19:23 +00:00
Bobby Buten a3e27bddac update js coverage with optimized jest builds
refs DE-989
flag=none

TEST PLAN:
  Confirm build still runs and coverage report is created

Change-Id: I800bf3a722b7bef13d2c7a42106827ce5f296d5d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/287828
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Bobby Buten <bobby.buten@instructure.com>
Product-Review: Bobby Buten <bobby.buten@instructure.com>
2022-03-23 19:43:58 +00:00
Bobby Buten 92e1839e88 optimize javascript build times
refs DE-1070
flag=none

TEST PLAN:
  Confirm build still runs
  Compare build times of pre/post changes

Change-Id: I9364314b8e5f99ea8f29cc69c0f711bf959c8fa3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/286171
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Bobby Buten <bobby.buten@instructure.com>
Product-Review: Bobby Buten <bobby.buten@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
2022-03-23 16:22:04 +00:00
Aaron Ogata d3ba157e43 remove translations/_core_en.js in production try 2
refs FOO-2720
flag = none

Short version: the "_core_en.js" file is no longer loaded in production
as its contents have been merged (there) with the super "core.js" file,
which is targeted for removal later in this series.

Change-Id: I05eb76a2f1cac12b4ab953bad0ca5a49b6ffd37b

---

The problem as I understand it: there are certain phrases that are
marked as "core" because they are used by shared/logic code like
DateTime formatters and are pre-defined in config/locales.yml. These
phrases were being supplied in two distinct files:

- _core_en.js, which includes only the phrases for the "en" language
- _core.js, which includes the phrases for the rest of the supported
  locales

_core_en was split because it was deemed necessary to be loaded always,
regardless of the active locale, and that is - as it appears to me, at
least - because some code attempts to look up those phrases at the
time their modules are evaluated. This patch corrects those modules to
defer their lookups until the point where the translations are used,
and when the translations have become available -- just like the rest of
the codebase does.

But if this is true, this begs the question, how come those modules
weren't presenting bugs by using translations for "en" and not for the
target locale? My thinking is that it is only coincidental that they
weren't: should _any_ module that uses I18n be evaluated _before_ any of
those modules, the Webpack plugin will have already loaded the "core"
file, which includes the translations for those phrases in the target
locale. _core_en may not have been loaded by then, but that doesn't
matter because the resolver is gonna look for what's in _core first when
it's available, and it is.

---

What happens in this patch is a slight change to prepare for the full
removal of both _core and _core_en: _core_en is now loaded only in
builds that don't load actual translations because we need the default
values that that file provides. The alternative would've been to go to
each call-site that looks up the phrases provided in _core_en and have
them supply default values, but it's untenable at this point.

This is the list of call-sites and the phrases they look up:

    ui/features/calendar/jquery/index.js: time.formats.tiny_on_the_hour
    ui/features/quiz_statistics/util/parse_number.js: number.format.delimiter
    ui/features/quiz_statistics/util/parse_number.js: number.format.separator
    ui/shared/day-substitution/backbone/views/DaySubstitutionView.coffee: date.day_names
    ui/shared/syllabus/jquery/calendar_move.js: date.month_names
    ui/shared/datetime/jquery/DatetimeField.js: date.formats.medium
    ui/shared/datetime/jquery/DatetimeField.js: date.abbr_month_names
    ui/shared/datetime/jquery/DatetimeField.js: date.day_names
    ui/shared/datetime/jquery/DatetimeField.js: date.abbr_day_names
    ui/shared/datetime/jquery/DatetimeField.js: date.datepicker.column_headings
    ui/shared/datetime/react/components/render-datepicker-time.js: datepicker.titles.hour
    ui/shared/datetime/react/components/render-datepicker-time.js: datepicker.titles.minute
    ui/shared/datetime/react/components/render-datepicker-time.js: datepicker.titles.am_pm
    ui/shared/handlebars-helpers/dateSelect.js: date.order
    ui/shared/handlebars-helpers/dateSelect.js: date.*
    ui/shared/i18n/i18nObj.js: number.format
    ui/shared/i18n/numberHelper.js: number.format.delimiter
    ui/shared/i18n/numberHelper.js: number.format.separator

dateSelect.js is the gnarly one because it seems to be passing through
everything under date.* to God knows who.

The list above was generated with a command similar to this:

    grep -rnP "I18n.(t|lookup)\(['\"](date|datetime|number|support|time)\S" ui

---

~ test plan ~
  ~~~~ ~~~~

- you can still activate a different locale and use something like the
  datepicker to normal effect

[change-merged]
[build-registry-path=jenkins/canvas-lms/foo-2720-01]

Change-Id: Ifd5d2d888edc9b89a9930824f2c55fd9c275b03f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284102
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
2022-02-18 00:08:38 +00:00
Aaron Ogata 0fbf4b027f convert JS job to EKS-compatible syntax
refs DE-687

Convert jobs over to EKS-compatible syntax in order to perform AB testing on EKS builds.

Test Plan
1. EC2 builds JS job with correct number of test results
2. EKS builds JS job with correct number of test results

[canvas-builds-refspec=3f9a35ef55d38ac885c3a4daef03e7e19cd8f4e7]

Change-Id: I350f2983eebeda4aa634daba0579389b3b987c6b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265580
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-06-07 23:05:24 +00:00
Aaron Ogata c52da6c883 move JS distribution to jsStage
refs DE-687

Test Plan
1. Test counts align with current master

Change-Id: I1900817b51f0857c6e22ea8361683cc08f61a19f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266448
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-06-07 16:07:31 +00:00
Aaron Ogata fe5d819183 stop passing delegate manually as function argument
refs DE-687

Test Plan
1. JS test count equals master test count
2. "expected to call" CPS warning doesn't appear

[canvas-builds-refspec=5a9ac2821ed203759e4f0e37f579774a210c81ba]

Change-Id: Id5e5c8c3b80ad5483f8b08913806b2195401e334
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266238
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-06-03 19:58:12 +00:00
Aaron Ogata 29e7aed396 merge all JS stages together
refs DE-687

Test Plan
1. All JS test counts align with latest post-merge

Change-Id: I2674f1dd594e561f1fbcc82d856342958ada19cc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265466
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-05-21 17:09:13 +00:00
Aaron Ogata 37da99de3c move js stages to shared library
refs DE-687

Test Plan
1. All JS test counts align with latest post-merge

Change-Id: Id803db3f13999e149ecb2dfed32efc66dac61c76
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265419
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
2021-05-21 15:20:26 +00:00
Aaron Ogata e76b7d4274 move docker copy operations to tearDownNode
refs DE-687

Test Plan
1. All test counts align with master for JS jobs

Change-Id: Iffe8ca4f873429b258269d371da3c67c1363a0e4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265395
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
2021-05-20 20:16:53 +00:00
Aaron Ogata 55442143fd align all JS output directories to single directory on Jenkins
refs DE-687

As part of our work on EKS, we need to align the EC2 / EKS paths to pull artifacts from the same path on the container and host. As a step towards that, ensure that all containers output their test files to the same directory.

Change-Id: I312fb5f8505122c68ba5aaba30730aa5c1887177
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265370
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-05-20 17:28:03 +00:00
Andrea Cirulli 70bb73c35b add jenkins file to the groovy linter stage
refs DE-690
flag = none

test-plan:
- Build passes
- Build trigger groovy linter stage if a change to Jenkinsfile is
made
- Build fails if a Jenkinsfile has linter issues

Change-Id: Ifdabb4f28f6124afcb6621844375a8de4b35d453
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265279
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2021-05-19 20:51:13 +00:00
Andrea Cirulli 3f057e9c0c fix jenkins files linter issues
Fix linter issues related to variable name, space after closing bracket.

refs DE-690
flag = none

test-plan:
- Build passes
- QA build with spec change executes fsc stage

Change-Id: I6f985fb3dfa8627703a27325b1eb25fae4585505
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265188
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2021-05-18 18:58:33 +00:00
Andrea Cirulli 67d6ccc360 fix jenkins file linter issues autofixable
refs DE-690
flag = none

test-plan:
- Build passes

Change-Id: Ie748a1e585ce8e3ec4978c7a5faa45dab524a905
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265184
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2021-05-18 16:48:15 +00:00
Aaron Ogata 4b5874d60f remove unnecessary indentation in JS job
refs DE-687

Test Plan
1. All JS jobs run correctly

Change-Id: I6b55907fd5bab5be598865ba2a0821a85863b81f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265091
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-05-17 19:51:51 +00:00
Aaron Ogata 073788ed9d movs JS setup / tear down functionality to new pattern
refs DE-687

Test Plan
1. Each JS stage executes all tests
2. Each JS stage build report summary works for failures

Change-Id: I2f9c9347ff6caac22ef3785e6a1a298870961f79
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265049
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-05-17 18:08:13 +00:00
Aaron Ogata c0d3dc25ee remove old coverage code from JS job
refs DE-462

As part of the EKS transition, we need to restructure the JS stages. The coverage code is currently not working and will get in the way of this transition. Remove it for now and refs the coverage restoration ticket so when we bring back the coverage build the code is here for viewing.

Change-Id: I7437584107ac13a79000d43f5c14baa31dac5027
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264864
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-05-12 22:19:42 +00:00
Kyle Rosenbaum 1e43e05241 add podTemplate and container definition to all applicable stages
refs DE-558, DE-582

[use-kubernetes]
[canvas-builds-refspec=24d3d83e8034ac6ad5380936498e1dcf01ab50eb]

modify the existing nodeRequirements to add podTemplate and
container definitions. initially, the podTemplate to be used
is docker-in-docker. the "use-kubernetes" flag must be set
as well.

test plan:
- build executes on k8s nodes in EKS when "use-kubernetes"
flag is set.
- build executes on EC2 when "use-kubernetes" flag is not set.

Change-Id: I13c68e5420bbf414532518eaf69ac82f9310bfbf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263257
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-04-27 19:47:25 +00:00
Kyle Rosenbaum 16469f7cfa add support for EKS container to protectedNode requirements
refs DE-558, DE-582

[canvas-builds-refspec=f9c80c0ac4b80de60ac50633523e30d5c2eb28b1]

modify the existing nodeRequirements call to pass a Map containing
the label as well as the possibility of container or podTemplate
params. this change only modifies the existing nodeRequirements to
continue using EC2 agents, not EKS.

test plan:
- builds still work as expected acquiring nodes

Change-Id: I0458a13a4c85e37aeb2f5dc56347e27101705646
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263208
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
2021-04-20 21:51:16 +00:00
Aaron Ogata b7b9d8cc4d remove redundant node clean-up code
refs DE-338

The protectedNode() function now handles automatic cleanup before / after a node is used, so much of the code here can be removed.

Test Plan:
1. Main build passes, FSC passes

[canvas-builds-refspec=81cc926fe89d6efd9f604d0c573630318fb4eb86]

Change-Id: Ie21d242aabe9826353f05149abb304a85223cebc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263149
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-19 16:37:23 +00:00
Aaron Ogata 5d1fb9b734 convert on*Result -> onStageEnded
refs DE-628

Test Plan:
1. Build summary report works for sub-jobs and normal stages
2. Sub-jobs run cleanupFn

[canvas-builds-refspec=82ccf7ae185871e45c658f225cc664ef05cda74d]

Change-Id: I9d4476b75e6488bcc388a89e9f7cf87150d0ae6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262951
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-15 13:54:20 +00:00
Aaron Ogata 3027dcdbd7 rename handler to hooks
refs DE-628

[canvas-builds-refspec=a5aca2425b1f885f5fcb4298479c0081d73065ff]

Change-Id: I0581d820fc521fb4857c04a28804be1c9a2d19a5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262947
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-14 22:50:58 +00:00
Aaron Ogata b3f6ee3b54 fix allow-stages directive
refs DE-621

Change-Id: I77e620defe993e37425cfd1f6e9c78d2574fb922
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262904
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-14 17:37:00 +00:00
Aaron Ogata 1a2b0052f1 convert extendedStage -> protectedNode to nodeRequirements() pattern
refs DE-621

Make the stage configuration aware of its node requirements for future EKS support and allowing nospot to be removed.

Test Plan:
1. RSpec / Selenium build works
2. Contract Tests works
3. JS works
4. FSC works
5. Vendored Gems works

[canvas-builds-refspec=480f3570a6a6ebee5cedaecbb3a089476a008215]

Change-Id: I6cb63df20abe8c1f913db4f1b21bb01bd4d30312
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262898
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-14 15:23:01 +00:00
Aaron Ogata 05dace17db remove cleanupFn usage from JS job
refs DE-624

As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.

Test Plan:
1. JS behaves correctly

Change-Id: Icbf8bd6d1f481573d58d61b55000ff601a7cd60f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262855
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-13 21:56:52 +00:00
Aaron Ogata e18887bf26 move docker cleanup to protectedNode() handler
refs DE-624

As part of making extendedStage() aware of stage node requirements, we need to remove consumers of the cleanupFn() functionality and replace them with extendedStage() compatible patterns.

Test Plan:
1. All stages execute correctly

[canvas-builds-refspec=07f1a75ee23f8f6a9a1bbe062316373b761e7a30]

Change-Id: Ica11b78ebead5c9239dea112677a077654c7090d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262814
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
2021-04-13 18:02:41 +00:00
Aaron Ogata 28454e524d work around flakey starlord token issue
refs DE-579

Change-Id: I0889b9e34c27a667ffa0573982bb77a37f103dc3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261023
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-03-18 18:20:52 +00:00
Aaron Ogata 7d73da5f3f disable translations for Jenkins build
refs DE-517

Disable locales on pre-merge Jenkins build in order to save ~25s on builds that require webpack to recompile. This also lets us clean up the code that builds the js-runner image. Pre-merge builds will still verify the i18n tasks in the Linters job, and post-merge builds will continue to build / run all tests using locales.

[build-registry-path=jenkins/canvas-lms/de-517]
[change-merged]

Change-Id: I8ded5d675b72d713738307403502f459adc79a73
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258197
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-02-08 16:03:06 +00:00
Aaron Ogata 70a0336523 use flakey network protection for js job
Change-Id: I9450cb2e94ac9542dde1219324cec5d348032c0c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258326
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-02-08 15:48:17 +00:00
Aaron Ogata a8102ab205 cache JS runner image
refs DE-490

Change-Id: I83f15e273f0b6fa059ab259780db4f3ac026805f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257869
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
2021-02-03 19:30:30 +00:00
Ahmad Amireh 46f8efd61f modernize canvas_quizzes
fixes FOO-1409
flag  = none

no more client_apps, canvas_quizzes now lives as part of canvas-lms
proper inside app/jsx/, which makes the build leaner and leaves us with
one less thing to reason about

logical changes:

- converted from AMD to ES modules
- upgraded to recent react + react-router
- dropped RSVP in favor of native Promises
- used CanvasModal instead of home-grown Dialog
- removed dead code; notifications in particular were fishy as there had
  no dependents at all and did not even show up in the graph
- ported tests to Jest, added more unit ones and two integration ones
- removed "config.onError" and now throws errors where appropriate
- disabled console statements in non-dev

:: test plan ::

- create a (old-school) quiz containing all types of questions
- as 3 distinct students, take the quiz and try to randomize your
  answers

at this point it's helpful to have a reference to compare the screens; I
replicated the quiz on my production sandbox for this

- go to /courses/:id/quizzes/:id/submissions/:id/log
  - verify it looks OK
  - click on a specific question in the stream and verify the question
    inspector widget works OK
  - go back to stream and push "View table"
  - verify the table and its controls are OK

- go to /courses/:id/quizzes/:id/statistics
  - verify it looks OK
  - click on ? in the discrimination index chart and verify it displays
    a dialog with help content
  - click on "X respondents" in one of the charts and verify it displays
    a dialog with the respondent names
  - verify the interactive charts do interact as expected (no logic
    changed here so just a quick glance)
  - link to "View in SpeedGrader" for essay-like questions works

Change-Id: I79af5ff4f1479503b5e2528b613255dde5bc45d3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256118
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-01-14 22:45:10 +00:00
Aaron Ogata 3e92cc4498 move execute function under libraryScript
refs DE-442

[canvas-builds-refspec=eecea52ff75a74be10cd0d2c1434dc917bd726af]

Change-Id: I7412d590e1c08a47eb48b1fd82af9eaf0d5eb145
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256507
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-01-12 00:32:08 +00:00
Andrea Cirulli 4d672c2f6a add global timeout and reevaluate all timeouts
Add timeout to all builds and set the proper values for the existing
timeouts

refs DE-307
flag = none

test-plan:
- Force the build to timeout and check that jenkins fails the build

Change-Id: I2e3ae36f828da1bc047bf5a2cfbc79c167b84d5f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256171
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2021-01-11 21:24:03 +00:00
Aaron Ogata 6593678c19 upload JS debug image
refs DE-413

Some users want to be able to load the built JS image locally for debugging purposes. We have 3 JS jobs that run in parallel and they all build the image separately. Create a 4th JS job that only runs if this image is requested in order to only upload the image once.

[upload-js-debug-image]

Change-Id: Iaa63971dcdf7bbae0c201a5fd0c8ffc319873a22
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254871
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-12-10 22:43:16 +00:00
Aaron Ogata 64b20f0c6a move coffeescript to own node
Change-Id: Ia8b95dbff31288b7e4e2b5fbf5839a92b2c5ecd8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253432
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-20 20:55:35 +00:00
James Butters cc45c56cb3 always checkout master code
we want to always checkout master code for testing on jenkins, unless
there are changes to our build scripts then pull the gerrit refspec.

flag = none

Test Plan:
- Normal Jenkins build passes.
- changes to docker-compose.new-jeknins* causes sub builds
  to checkout code from gerrit refspec.
- changes to any file in build/new-jenkins/ causes sub
  builds to checkout code from gerrit refspec.
- Normal build with no changes to Jenkinsfile*, build/new-jenkins/,
  or docker-compose.new-jenkins* sub builds checkout master.

Change-Id: If1224a44b52b00f0f056b9552cd562809fc7eaa8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252999
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
2020-11-18 21:48:19 +00:00
Aaron Ogata 5344111714 restructure JS jobs for ruby-runner change
refs DE-376

With the ruby-runner change that is coming, the final image will no longer include the node_modules/ assets, so we have to checkout the WEBPACK_BUILDER separately and copy in the source from the final PATCHSET_TAG.

Change-Id: Ic3c048500e00858bfa39244b308112cbc721783d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252666
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-17 19:25:41 +00:00
Aaron Ogata f6e7acb991 Revert "restructure JS jobs for ruby-runner change"
This reverts commit d893385149.

Reason for revert: broke Jenkins

Change-Id: Ic3fe2f4d4940cbad99ab6074741fa7b73ba774a0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252665
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Tested-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-17 17:05:11 +00:00
Aaron Ogata d893385149 restructure JS jobs for ruby-runner change
refs DE-376

With the ruby-runner change that is coming, the final image will no longer include the node_modules/ assets, so we have to checkout the WEBPACK_BUILDER separately and copy in the source from the final PATCHSET_TAG.

Change-Id: Ide3885f41d7ee21f4034331c204c2cafdc76aa28
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252922
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-17 15:33:33 +00:00
Andrea Cirulli ba4ad02c38 abstract checkout code in jenkinsfiles
refs DE-351
flag = none

test-plan:
- Build passes
- Test FSC is working

Change-Id: Id3844dd12600e5d3a80d267bf67916c4d43d2548
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2020-11-04 20:58:00 +00:00
James Butters 986c935bdf checkout Jenkinsfile from master
These changes need to be accompanied by changes
to Jenkins config:
  - main-canvas-postgres-12-ruby-2.6 and
  main-canvas-postgres-12-ruby-2.6-postmerge, SCM
  refspec changed to master
  - sub-builds scm changed from canvas-lms-refspec
  to JENKINSFILE_REFSPEC defaulting to master
  - add Jenkins parameter JENKINSFILE_REFSPEC to sub-builds
  - add new Jenkins build for main Jenkinsfile changes
  - main build don't trigger on Jenkinsfile changes using
    forbidden path.

Testing once merged:
- verify builds without Jenkinfile changes pull the master
  version of Jenkinsfile
  - verify sub-builds as well
- verify gerrit with Jenkinsfile changes triggers new build
  - verify sub-builds get Jenkinsfiles from gerrit not master
  - test a combination of Jenkinsfiles, not just the main one
  - verify main build is not triggered
- run a plugin build without canvas-lms-refspec
- run a plugin build with canvas-lms-refspec

Change-Id: Ic6be8b446f80d57f7b0f25c77e6cffb959e03236
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251634
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
2020-11-03 03:08:08 +00:00
Aaron Ogata 2dcc3df114 introduce canvas-builds-refspec commit flag
refs DE-349

[canvas-builds-refspec=f04c7902ccabadb61cee13fa93f09b1184925b84]

Test Plan:
1. Build pulls master when flag does not exist
2. Build pulls specified commit when flag exists
3. Build pulls master for post-merge when flag exists

Change-Id: I365efd3a305d08f7a3df76fd44799756104b4c1f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251635
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-02 19:33:54 +00:00
Andrea Cirulli 9127a73426 add timeout to cleanup function callback executed by protectednode
in some cases if the node removal happened after the code checks for a
node removal event the cleanup steps are executed and they can cause
extremely long running build. A fix to that is the introduction of a
timeout in the cleanup function.

refs DE-306
flag=none

test-plan:
- Canvas-lms build passess

Change-Id: Ia739bbdc1b72eac8ae7e95d5d36ebd698ce6d56e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249252
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
2020-10-05 19:26:58 +00:00
Aaron Ogata dbf584aafb fix JS failures
After upgrading to Ubuntu, Chrome takes up more memory than it does on Alpine, and the node becomes overloaded, resulting in timeouts. Reducing the coffeescript split to 1 node results in no timeouts after 40 test runs with no speed regression.

refs DE-260

Change-Id: I45efbfecb1a2fc25a7d89f91de925f2b6e84f25a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247622
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-09-14 16:24:01 +00:00