Commit Graph

22 Commits

Author SHA1 Message Date
Ahmad Amireh 5f976f9989 rename frontend_build -> ui-build
minor QOL mainly for me; put webpack-specific configuration in its own
folder, make it portable, and make room for esbuild

the __webpack_public_path__ initializer no longer sources the value from
the build module, instead it uses a pre-defined global that webpack
injects

~ test plan ~

build is OK

Change-Id: I4ba4a3c0cb9175f96096f2b78022e152c04fc75d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276183
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
2021-10-21 19:13:02 +00:00
Cody Cutrer 31c7af578f remove SafeYAML
Psych has safe_load now, and it's fairly trivial to convert our existing
overrides to use that instead

Change-Id: I2648df8d4574e15fc9072a25882e318d902765c3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271939
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-24 16:58:54 +00:00
Ahmad Amireh fd8a4aebd8 remove plugin symlinks
fixes FOO-1824
flag = none

[pin-commit-analytics=7f2f049830a11e43f4a5d0031e3c4ecc3145ccdb]
[pin-commit-instructure_misc_plugin=9b505159130c0724ce6bcbb75d07d40ff16a1961]

read directly from gems/plugins/* instead of symlinks

- dropped dead brandable_css config property "all_sass_files"
- webpack test runner now generates a file that runs all plugin spec
  files (from spec_canvas/coffeescripts)

test plan: you can remove your symlinks and still survive!

Change-Id: I0206c2d827aa9f59b0374b21f0863443dff3be0f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262346
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-05-20 17:21:08 +00:00
Ahmad Amireh 4243097b21 rename some things
fixes FOO-1265

[skip-eslint=true]
[pin-commit-analytics=7e49eefd7f59cbf43fd03bf8957bbbfa76f8d9d5]
[pin-commit-demo_site=adade2e38e46a358a4643cd3db2fe5ccffe39ec5]
[pin-commit-instructure_misc_plugin=47a3161102b0611af56d134fbd0c828ddc1b8abf]
[pin-commit-migration_tool=0dbac2f5b421d894395605ce4b583ef0f7d60b22]
[pin-commit-multiple_root_accounts=cc96d28c0d59bbe47acc64de4fdd8e1d2b029805]

Change-Id: I14c07f20bd260cf0de1f48ceb70b3c2313edf2d2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258807
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
2021-04-06 01:12:49 +00:00
Aaron Ogata 2b4747c4b5 cache docker/dockerfile to allow flakey network script usage
Change-Id: Ide628034a970e2e6fded7c2971712d9dfed6391e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262253
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-04-05 15:39:21 +00:00
Aaron Ogata d7e131b296 mirror busybox to starlord
refs DE-563

Change-Id: Ia48d1c078159aa3c0e3860cbb33ca34afcee8483
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260658
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-03-15 18:26:04 +00:00
Ahmad Amireh 5a9a4f3470 (re) decentralize i18nliner configuration
refs FOO-1408
flag = none

This reverts commit c904d83673.
Original commit 9af1badd2f.

I've also made an adjustment where you may only include a directory and
not a specific file so that we don't make it possible to configure
outside of an ".i18nrc" file (a measure to eliminate surprises that's
all)

before:

    { "include": [ "path/to/dir/.i18nrc" ] }

now:

    { "include": [ "path/to/dir" ] }

:: test plan

run "rake canvas:compile_assets" and visit any page with JavaScript
text, that text should display correctly

Change-Id: I320ec4056270647835035f45501d310b18d9cb25
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258178
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
2021-03-10 18:52:49 +00:00
Ethan Vizitei b33b74151b stage 1: audits engine extract
refs FOO-1644
[fsc-propagate=false]

  - create engines directory
  - put one empty-ish engine in there
  - add general engines dependency to
     gemfile construction
  - update build pipeline to run tests all engines
     just like gems
  - add engines directory to jenkins-cache build
  - get rspec running successfully with a dummy app
     inside the audit engine
  - stop failing with no message if environment
     unloadable in css config
  - generally set pattern in build artifacts and in
     code organization for pulling out an engine

Change-Id: I8650f5e0aa7d72556d2f02156886d2b503a111fb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258988
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-03-04 20:27:06 +00:00
Ethan Vizitei cf33fba96c extract ConfigFile to gem
refs FOO-1627
flag=none

make Dockerfile cache use
gem instead

TEST PLAN:
  1) specs run successfully
  2) config in operational environments loads successfully
  3) reloading settings still causes config files to get
     re-parsed

Change-Id: I69bdfc2b79065600336f83c7ffb87dd8e647fa9c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259232
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2021-02-23 21:18:20 +00:00
Cody Cutrer f67d08e13f default to rails 6.0
you can switch back to rails 5.2 one of these ways:
 * ENV CANVAS_RAILS6_0=0
 * consul key rails6.0 with value `false`
 * touch config/RAILS5_2

Change-Id: I811884b2250b865220a1a9d84c7254f42d79e18f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257587
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
2021-02-08 23:32:22 +00:00
Jacob Burroughs c904d83673 Revert "decentralize i18nliner configuration"
This reverts commit 9af1badd2f.

Change-Id: I159a478d1f3baa062148e3fd5ff4ee914176579c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256982
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-01-19 19:52:58 +00:00
Ahmad Amireh 36920ba9e1 remove "exports" from all @instructure/ packages
refs FOO-1331
refs LS-1657
flag = none

test plan: canvas-rce mocha tests work after upgrading to inst-ui 7

Change-Id: Ibe8d94fdd5a994fda2c7b9703dccfb77a62415ca
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256766
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
QA-Review: Ed Schiebel <eschiebel@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-01-15 19:39:02 +00:00
Ahmad Amireh 46f8efd61f modernize canvas_quizzes
fixes FOO-1409
flag  = none

no more client_apps, canvas_quizzes now lives as part of canvas-lms
proper inside app/jsx/, which makes the build leaner and leaves us with
one less thing to reason about

logical changes:

- converted from AMD to ES modules
- upgraded to recent react + react-router
- dropped RSVP in favor of native Promises
- used CanvasModal instead of home-grown Dialog
- removed dead code; notifications in particular were fishy as there had
  no dependents at all and did not even show up in the graph
- ported tests to Jest, added more unit ones and two integration ones
- removed "config.onError" and now throws errors where appropriate
- disabled console statements in non-dev

:: test plan ::

- create a (old-school) quiz containing all types of questions
- as 3 distinct students, take the quiz and try to randomize your
  answers

at this point it's helpful to have a reference to compare the screens; I
replicated the quiz on my production sandbox for this

- go to /courses/:id/quizzes/:id/submissions/:id/log
  - verify it looks OK
  - click on a specific question in the stream and verify the question
    inspector widget works OK
  - go back to stream and push "View table"
  - verify the table and its controls are OK

- go to /courses/:id/quizzes/:id/statistics
  - verify it looks OK
  - click on ? in the discrimination index chart and verify it displays
    a dialog with help content
  - click on "X respondents" in one of the charts and verify it displays
    a dialog with the respondent names
  - verify the interactive charts do interact as expected (no logic
    changed here so just a quick glance)
  - link to "View in SpeedGrader" for essay-like questions works

Change-Id: I79af5ff4f1479503b5e2528b613255dde5bc45d3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256118
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-01-14 22:45:10 +00:00
Ahmad Amireh 9af1badd2f decentralize i18nliner configuration
fixes FOO-1408

this allows us to control which files i18nliner processes completely
through config and without having to modify hardcoded paths in source
file, which is something we need for FOO-1265

.i18nrc files can include other directories through the "include"
directive:

    // file: canvas-lms/.i18nrc
    { "include": [ "public/javascripts/.i18nrc" ] }

    // file: public/javascripts/.i18nrc
    { "files": [...] }

:: test plan

aside of Jenkins exercising the i18n tasks, I ran a diff by hand over
the set of files that i18nliner processses before and after the patch,
with the new code processing a few more files: some handlebars in the
analytics plugin and the 3 client_apps/canvas_quizzes source files

if you really want to, you can do the same or find another way to verify
the output

on master, edit canvas_i18nliner/js/main.js somewhere before the
exports:

    Check.prototype.checkWrapper = f => console.warn(f)

run it:

    ./gems/canvas_i18nliner/bin/i18nliner check 2>
      tmp/i18nliner-upstream-files.txt

    cat tmp/i18nliner-upstream-files.txt | sort >
      tmp/i18nliner-upstream.txt

now do similar on our branch (although we need to massage teh output
because the paths are absolute:)

    ./gems/canvas_i18nliner/bin/i18nliner check 2>
      tmp/i18nliner-patched-files.txt

    cat tmp/i18nliner-patched-files.txt | sort >
      tmp/i18nliner-patched.txt

    sed -i "s{$PWD/{{" tmp/i18nliner-patched.txt

now look for differences:

    git diff --no-index \
      tmp/i18nliner-upstream.txt \
      tmp/i18nliner-patched.txt

Change-Id: Ic73cbc7261ab597deb567fc5d0af1e3014875da1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/255952
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-01-07 15:58:18 +00:00
Ahmad Amireh 45cb519d52 [node12] upgrade to node 12.19.1 (LTS)
fixes FOO-1116
flag = none

this upgrade was problematic mostly because of Node's support for ES
modules (ESM) that is still fairly recent and introduces some complexity
around packaging. This is also the reason for the considerable hack
related to @instructure packages (see inline documentation) that can be
removed once we fix things upstream

| test plan |
| --------- |

- install node 12 if needed (e.g. not on docker)
- purge your node modules:

    rm -rf {.,client_apps/*,gems/*,gems/plugins/*,packages/*}/node_modules

- make sure you can still build locally (native/docker)

    bundle exec rake canvas:compile_assets

Change-Id: If9605ac428f0ff228f8852f0becb9fbac750f1f2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253166
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
2020-12-11 03:45:31 +00:00
Aaron Ogata d2d12971e7 generate pre-computed packages cache
refs DE-376

For the ruby-runner change, we will need to have the yarn packages dependencies pre-computed in a separate docker image so we can collect the MD5SUM of those changes. Also, the final ruby-runner image will not contain the packages/ directory, so we have to move it to the webpack-builder image for the JS jobs.

Change-Id: I37698e06090dcbee815651120f92bf39538e886f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253584
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-23 23:29:14 +00:00
Aaron Ogata 11d2366ebb optimize cache helper images
refs DE-376

Change-Id: Iaf7ba66d2e571614e7ec1feba6fced72e0714dbc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252800
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-11-13 19:25:46 +00:00
Aaron Ogata 0070f199be share dependencies cache image between builds
refs DE-370

[change-merged]
[build-registry-path=jenkins/canvas-lms/de-370]

Test Plan:
1. pre-merge build uses cache
2. post-merge build uses cache for post-merge image
3. post-merge build uses cache for pre-merge image
4. post-merge build reuses webpack-builder for pre-merge

Change-Id: Ie5d5eaad2d9da1f324a87020cb70658d42b0bf94
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252353
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
2020-11-10 19:27:39 +00:00
Aaron Ogata e15da9e052 tighten config/ scope of webpack cache helper image
refs DE-324

Change-Id: Ib370313445edbf6635cfdbbcbfd886a8b1b6229f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250796
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
2020-10-22 13:58:44 +00:00
Aaron Ogata eaa56c4fa7 tighten lib/ scope of webpack cache helper image
refs DE-324

Change-Id: I6c078c62b8bc77983077f5edf7ad2a764b6bca37
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250686
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-10-22 13:19:56 +00:00
Aaron Ogata e20e187681 tighten scope of webpack cache helper image
Change-Id: I7ca2b6b3652940b83e9f0c8bb11defed0c4c7e95
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250575
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-10-21 13:35:30 +00:00
Aaron Ogata 155402f15d remove buildkit for final Jenkins image
Buildkit seems to have a race condition where the cache is not used, or incompletely used, mostly on a new instance, but sometimes on an instance that has already used the cache successfully. We created a partial reproduction (g/250409) where with buildkit, layers above the modified layer were rebuilt, and with this change, all cached layers are correctly used.

[change-merged]
[build-registry-path=jenkins/canvas-lms/final-no-buildkit]

Change-Id: If10c186583cc21421d861bc2d1d43c7d2b0fc2f9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250400
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2020-10-19 15:51:33 +00:00