Change-Id: Ic9bb3deea7d89a48cd23e0ee8cd138cdd9e89321
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/344092
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
Build-Review: Isaac Moore <isaac.moore@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
flag=none
test plan:
- Specs pass.
- Make sure you can run canvas using docker
with this new version of the bundler.
- If you are using inst-cli, you can do:
inst canvas clean
inst canvas setup
qa risk: low
Change-Id: I259119d6ec09332480f0e110d22fa233c3ac1b7e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/338012
Reviewed-by: James Butters <jbutters@instructure.com>
Build-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: Omar Soto-Fortuño <omar.soto@instructure.com>
Change-Id: I799dc922a081745102d876189e4da56ed0a2533c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/326546
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
Build-Review: Isaac Moore <isaac.moore@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
also, don't set BUNDLER_VERSION env var in Docker, so that
bundler's auto-install-and-restart-with-the-correct-version
feature can work
Change-Id: I8e3722197fb3598b5c40679d997f19b3b3957ea8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/326580
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Build-Review: Cody Cutrer <cody@instructure.com>
refs AE-297
and also include binstubs in the path in docker
this will help ensure people are running the correct version of bundler
Change-Id: I68e7f6a43efd89ff257a9307aa4f6709acef05f8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/318776
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Build-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
When building the canvas web container, references to the arg RUBY
are not set resulting in unexpected file paths. For example
GEM_HOME only creates /home/docker/.gem/ when we expected it to
create /hom/docker/.gem/2.7 by default. We need to add another
reference to the ARG RUBY after the FROM.
According to Docker: An ARG declared before a FROM is outside of a
build stage, so it can’t be used in any instruction after a FROM.
To use the default value of an ARG declared before the first FROM
use an ARG instruction without a value inside of a build stage
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
Test Plan:
-run `docker compose build web`
-bash into the web container
-echo $GEM_HOME
-path is set to /home/docker/.gem/2.7
-run `docker compose build --build-arg RUBY=3.1 web`
-bash into the web container
-echo $GEM_HOME
-path is set to /home/docker/.gem/3.1
Change-Id: I4856a8ffff3debe9deac3f3fb511d13735f41fea
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/314062
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
Change-Id: I3c1fd74154a52aad96ef7ab85144dcc8dceb414a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/310751
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Build-Review: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
Change-Id: I954402cb7016c1590748f149279865db967272a3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/308110
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Build-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
[canvas-builds-refspec=5bdd0f9fb485ef9501bd0f34f8f669b36fa02c30]
[postgres=14]
[skip-cache]
postgis appears to be unused, so removed. Upgrade to use pg14
to match production.
Test Plan:
- Jenkins passes
- docker setup scripts work for amd and arm
Change-Id: I57e87b6a0d800d9574b2d1b7042d9b7939dc8b73
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/299570
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Build-Review: James Butters <jbutters@instructure.com>
While I am at it, decaffinate and typescriptify TextHelper
fixes FOO-2987
Change-Id: I0b8f33a610bee737ea79ad24d8aec8f8e8051c49
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/299013
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Build-Review: Bobby Buten <bobby.buten@instructure.com>
Change-Id: I1e6aae06c904d2fd57f2720057c7a1f1629680f2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/295294
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
refs DE-523
Test Plan
1. Ensure CANVAS_RAILS override works
2. Ensure file override works
3. Ensure consul override works
Change-Id: I6eea63bba5c401428c02179a3d6187c8265ce33e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/287849
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
in preparation for migrating qti_migration_tool to python 3
once that migration is complete, we can remove the python 2 version
refs DE-993
Change-Id: Iea25e7418c6e4bc87e639cc1883b216eea94ca6c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282813
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
refs DE-966
flag=none
TEST PLAN:
ensure build completes and all tests run successfully
Change-Id: I82ef402eb790fafce53393b17b4695c9151e1745
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/281489
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ben Rinaca <brinaca@instructure.com>
QA-Review: Bobby Buten <bobby.buten@instructure.com>
Product-Review: Bobby Buten <bobby.buten@instructure.com>
We're back to makign installing pulsar the default again. Later test
builds broke without it.
Change-Id: Icb994b5dc5a566537c4c9ad513ab0f0b2ca2893e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/281209
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Keith Garner <kgarner@instructure.com>
Product-Review: Keith Garner <kgarner@instructure.com>
In earlier attempts to make pulsar optional for our docker builds, the
dockerfiles that need to be in sync got out of sync. This returns them
to being in sync, and makes the pulsar gems optional which is better for
open source users.
Change-Id: Id8a02878bb61cf589670526da0626a90da530a58
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/281101
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Keith Garner <kgarner@instructure.com>
Product-Review: Keith Garner <kgarner@instructure.com>
refs DE-545
When apt-get tries to pull a package from a server that is down or undergoing maintenance, builds that modify Gemfile can fail due to not being able to pull packages. By splitting the package installation into its own layer, external packages no longer need to be reinstalled when gems are updated.
Test Plan
1. Cache builds correctly from scratch, uploads all relevant images.
2. Cache is correctly reused when no gems are updated
3. Base cache image is reused when gems are updated, and all other images are rebuilt
4. Pre-merge build passes and uses cache as appropriate
5. Updating Dockerfile.jenkins re-builds base image
[build-registry-path=jenkins/canvas-lms/de-545-1]
[change-merged]
Change-Id: Ifd76064892817abb80dd0daebe8c2189c0338d78
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279697
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
with the recent upgrade of the ruby version in our Jenkins build
we need to make our Dockerfiles match that version 2.7
[canvas-builds-refspec=2427eaa818279a3b95f5ae0169d4896ba0cb5f7d]
flag = none
Test Plan:
-Jenkins passes
-local docker dev setup still works, no regressions
Change-Id: I4a87aa401df1c8c58b733990aba619aeed2e6347
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268900
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs FOO-1116
right now in order to build the javascript, we must run brandable_css
first for the specific purpose of injecting into handlebars templates
their corresponding stylesheet file with the checksum as reported by
brandable_css
this patch moves things around such that Rails provides this mapping of
"handlebars template" to "CSS variant file checksums" in a global
variable (Rails is privy to this information already), webpack then at
build time injects only a reference to that mapping into the template,
then our runtime brandable-css JS helper will resolve that reference
from the mapping and request the correct file based on the current
user's variant
[1] rails assigns window.BRANDABLE_CSS_HANDLEBARS_INDEX with content
like this:
[
[ "new_styles_normal_contrast", "new_styles_high_contrast", ... ],
{
"0c": ["0f4f01ab86", 0, 0, "e3e7211701", 3, ... ],
...
}
]
the structure of the mapping is explained in brandable_css[1], it is
a bit overtuned to reduce the size as much as possible since this is
included for ALL pages even though the templates in question may never
be rendered (right now it's ~1.5kB and should never grow)
[2] webpack asks @canvas/brandable-css to loadStylesheetForJST using the
contrived id for that file (frontend_build/i18nLinerHandlebars.js:92)
[3] brandable-css/index.js will decode that structure to arrive at the
exact URL for the stylesheet for that template for the current user
variant
with this , we should be able to optimize "rake canvas:compile_assets"
to run both brandable_css and webpack in parallel, which will greatly
mitigate the performance decrease from moving from node-sass to
dart-sass
== test plan
go to some assignment's syllabus page, edit the file
app/stylesheets/jst/courses/Syllabus.scss with something fun like:
#syllabus .day_date { background: $fire; }
rerun `npx brandable_css`, reload the page and verify it's on fire,
optionally change your setting to enable High Contrast mode, reload
the syllabus page and verify the color changes (it picks up the other
variant stylesheet)
[1]: 80af6e70af
Change-Id: Ie48cc71d4837ee02313b2d45a9ad2944725acad2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268615
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
This reverts commit 655cecbd5d.
Reason for revert: broke dev environments
Change-Id: I97f47a8ef7ac369b3cb30e448a3cfa9997c03930
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267650
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
move asset compiling into the Dockerfile allowing for a simpler
docker dev setup.
flag = none
closes: DE-696,DE-365
Test Plan:
-mac normal setup
-mac mutagen setup
-linux normal setup
-linux mutagen setup
Change-Id: I7e178c1ca91ee1908c4af55e03ec0010f5abce8e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266332
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
because simply asking it to run yarn-deduplicate is no longer good
enough, you must now be polite when doing it[1]:
To prevent security and user-experience problems from mistyping
package names, npx prompts before installing anything. Suppress
this prompt with the -y or --yes option.
we will also be updating npm to the latest to catch such issues in the
future
== test plan
running "rake canvas:compile_assets" using npm > 7 completes
without hanging indefinitely at the "install dependencies" stage
[1]: https://docs.npmjs.com/cli/v7/commands/npx#compatibility-with-older-npx-versions
Change-Id: I81e044d20143991cf6103084f44ce8d4c9011f5b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259662
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
because of nokogiri, we rely on multi-platform gem caching, which
only works right in bundler 2.2
Change-Id: Id207278946e849abec3418807c71e4650506a0cf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259142
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
you can switch back to rails 5.2 one of these ways:
* ENV CANVAS_RAILS6_0=0
* consul key rails6.0 with value `false`
* touch config/RAILS5_2
Change-Id: I811884b2250b865220a1a9d84c7254f42d79e18f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257587
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
fixes FOO-1527
flag = none
test plan: you can install node modules and run webpack
Change-Id: I33347d791b0923f4471b555c6fac303b098b1ef4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257333
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs DE-368
When the images are rebuilt, it changes the image digest causing the cache to become invalid. It can also break the build and is not easily revertable to the previous image. Work around this issue by mirroring the dockerhub image on starlord, and update it via a cron job.
Change-Id: I2f3f80ea53c7c306a126805b429781743df85aa6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256167
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Add a ruby 2.7 build triggerable by a comment on gerrit
refs DE-387
flag = none
test-plan:
- Build passess
- Local dev builds
- Ruby 2.7 build uses the ruby2.7-passenger image for
building the application
Change-Id: I512df9344dbd77e4c592270ef260325f95d8f666
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/255797
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
fixes FOO-1116
flag = none
this upgrade was problematic mostly because of Node's support for ES
modules (ESM) that is still fairly recent and introduces some complexity
around packaging. This is also the reason for the considerable hack
related to @instructure packages (see inline documentation) that can be
removed once we fix things upstream
| test plan |
| --------- |
- install node 12 if needed (e.g. not on docker)
- purge your node modules:
rm -rf {.,client_apps/*,gems/*,gems/plugins/*,packages/*}/node_modules
- make sure you can still build locally (native/docker)
bundle exec rake canvas:compile_assets
Change-Id: If9605ac428f0ff228f8852f0becb9fbac750f1f2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253166
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
[change-merged]
[build-registry-path=jenkins/canvas-lms/ruby-runner]
Test Plan:
1. Ensure that changing gems updates the whole image.
2. Ensure that changing npm updates the webpack builder image.
3. Ensure that changing webpack files are reflected in the final image.
4. Ensure that changing packages files are reflected in the final image.
5. Ensure that there are no speed regressions in fully-cached cases.
6. Ensure that caching works as expected.
Change-Id: Icef77195ff912bc0628f27eeda6780f262a8908a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252565
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-376
For the ruby-runner change, we will need to have the yarn packages dependencies pre-computed in a separate docker image so we can collect the MD5SUM of those changes. Also, the final ruby-runner image will not contain the packages/ directory, so we have to move it to the webpack-builder image for the JS jobs.
Change-Id: I37698e06090dcbee815651120f92bf39538e886f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253584
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Cody Cutrer