refs FOO-2520
flag = none
[pin-commit-multiple_root_accounts=2a9bf89895f38df6bf8f54828af66aced594abf0]
revisit the API for resolving asset names to their (real)path on disk,
because adding to the existing logic to support an alternative bundler
made things hard to understand.
This patch brings a new simplified interface Canvas::Cdn::Registry to
query assets and resolve their location.
- Registry#include?(path) tells whether a realpath points to a static
asset
- Registry#statics_available? tells whether static assets are available
- Registry#scripts_available? tells whether JS assets are available
- Registry#scripts_for(bundle) provides the realpaths to all the JS
files in the specified bundle
- Registry#url_for(name) provides the realpath to the static asset
The Registry is a good place to house the BrandableCSS resolving logic
in the future for even more consistency. It can also support an
alternative bundler internally without leaking. Eventually, it would be
nice to have it as a gem.
CHANGES
-------
- helper "font_url_for()" has been removed as it was a duplicate of
existing logic; instead use "font_path(...)" to achieve the correct
result. As a result, BrandableCSS is no longer querying Gulp's
manifest.
- preloaded fonts are now aware of the asset host and work for CDN
- InfoController uses the new Registry API to tell whether Gulp and
Webpack have produced their assets successfully
- ApplicationHelper no longer re-computes the base URL for JavaScripts,
now only the Registry is concerned with that
- ?optimized_js query parameter is no longer supported as it has no real
benefit now that we have access to sourcemaps on production
- ENV['USE_OPTIMIZED_JS'] is now more consistent as there is a single
source of truth for it. The Registry can be instantiated with
{environment: "production"} to point to the optimized version of the
scripts.
- "css:compile" task no longer writes BrandConfig records to the DB,
that is now done as part of the "compile_assets" task, which you can
opt out of doing by setting COMPILE_ASSETS_BRAND_CONFIGS=0
TEST PLAN
---- ----
- load your dashboard and verify all the assets are loaded correctly
- set up a CDN, restart your Rails server and reload the dashboard
- verify all assets are loaded from the CDN
- verify the Lato fonts are pre-loaded from the CDN
- (optional) add custom JS to a sub-account and visit it
- verify the custom JS is loaded and evaluated *after* Canvas's main
javascript bundles
Change-Id: I8198de747cdd5892d6a831cb6c61ba0ef9afa789
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276537
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
fixes INTEROP-6418
flag=none
Test plan:
- create a dev (API) key. Set it to enforce scopes but just add some
scope we won't test, such as Account Notifications. For redirect URI
use something like "http://test.docker/foo"
- Go to Account -> Settings -> Edit Settings and change your language to
your favorite non-English language. The ticket suggests German
(Deutsch)
- run the first step of an Oauth2 flow. This is a GET request to
/login/oauth2/auth with the following query params (example):
?client_id=10000000000016
&response_type=code
&redirect_uri=http%3a%2f%2ftest.docker%2ffoo
&scopes=url%3AGET%7C%2Fapi%2Fv1%2Fcourses+url%3APUT%7C%2Fapi%2Fv1%2Fcourses%2F%3Aid+url%3AGET%7C%2Fapi%2Fv1%2Fcourses%2F%3Aid+url%3ADELETE%7C%2Fapi%2Fv1%2Fcourses%2F%3Aid
- The oauth endpoint will forward you to a page with an error in the URL
(as a query param). the error message should say "The following scopes
were requested, but not granted:" with a list of scopes including the
English word "and", not the German word "und".
- check out the commit above this and repeat the Oauth2 flow to make
sure that it was indeed broken (translates the word "and" to the
foreign language)
Change-Id: I8f64c7b6b5f048d38d539da9d8067e4d94156717
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282069
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
[skip-stages=Flakey]
auto-corrected with post review ensuring line breaks continue to
convey original meaning of separated conditions, and cleaning up
some now-duplicated conditions
Change-Id: Ib9b31226de0665a2e4427fe595639d0d91a33f83
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279151
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
auto-corrected, with a few instances of reworking things that modify
the hash as they're iterating
Change-Id: I73cef3016e46309c1c2ce906b013288d73976acf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278781
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
auto-corrected, with post review to remove unnecessary empty down methods
in migrations, and change def x(*args); end to just def x(*); end
Change-Id: Ic006bcebb0b073e6c66ed957a561c93c3d368e24
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278893
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Migration-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
auto-corrected
also enable Style/PercentQLiterals forcing uppercase with it, so that
both %q and %Q get changed to bare %
Change-Id: I91389c18d864b3ec638c6cd366c2c74f78c69a57
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278673
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
auto-corrected, with post-review changing hashes that were meant
to be string keys in the first place
Change-Id: I877a365b9035bb62cea4d3b2f01f641f55b63281
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278676
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs QUIZ-8266
flag = none
this PS should be tested with g/277137 g/278455
test plan:
- create a classic quiz that contains questions and stimuli
- migrate the quiz
- access the quiz_api console
- observe that the migrated quiz and its items and stimuli
have their migration_id fields populated
Change-Id: Ibf6ae46685daca16b2f546ec8f10475a91c1fe14
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277070
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
Reviewed-by: James Logan <james.logan@instructure.com>
Product-Review: Jorge Arteaga <jorge.arteaga@instructure.com>
why:
* to manually send one of these every minute through the live events
ecosystem, for testing
refs INTEROP-7143
flag=none
test plan:
* specs
Change-Id: I860bdab0af9bb363df63ae291da5a900231b4044
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277281
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
[skip-stages=Flakey]
Naming/HeredocDelimiterNaming and Rails/SquishedSQLHeredocs
the former was manual, the latter was automatic. I also changed
some <<- to <<~ to allow for better formatting
I also had to change comments inside squished SQL heredocs to
be block comments (since newlines are removed); searching for those
I found some multi-line strings that are better as heredocs
Change-Id: I6b138f8e32544b97df1e4c56f09ee5316cbdef9d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278184
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
auto-corrected, with review for non-single-argument-callers.
there weren't any, but I did disable it anyway for one file where
it seemed to make the DSL less legible (QtiItems)
Change-Id: I1b4c43ffd899e656902981baac213ca394791b67
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278156
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
auto-corrected, with manual review to identify possible nilness
Change-Id: I205436e5c3cb37aae99ea552c7d14e6d1a04ef06
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277893
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
all manual
the fixes are a little scattered, since the same method doesn't work
everywhere depending on requirements. mostly I changed to `let`, but
some required `stub_const`. For `let`, I eventually settled on
avoiding a dedicated `let` for the class if it's only used one, and
it's a trivial class just to include the module. otherwise there's
a separate `let` for the class, and if there's only one it's named
`klass` instead of something contrived.
Change-Id: I84734c963d4789be3ec3cd852cca623e7c2a08df
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277285
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-2410
test plan:
- in dynamic_settings.yml, add the following block:
```
store:
canvas:
services-jwt:
# these are all the same JWK but with different kid
# to generate a new key, run the following in a Canvas console:
#
# key = OpenSSL::PKey::RSA.generate(2048)
# key.public_key.to_jwk(kid: Time.now.utc.iso8601).to_json
jwk-past.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-05-18T22:33:20Z_a\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
jwk-present.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-06-18T22:33:20Z_b\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
jwk-future.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-07-18T22:33:20Z_c\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
```
- Ensure /internal/services/jwks loads correctly
- In console, ensure `CanvasSecurity::ServicesJwt.decrypt(Base64.decode64(CanvasSecurity::ServicesJwt.for_user('localhost', User.first)))`
and `CanvasSecurity::ServicesJwt.decrypt(Base64.decode64(CanvasSecurity::ServicesJwt.for_user('localhost', User.first, symmetric: true)))`
both work and produce sensible looking output
Change-Id: I13c6c35cc92ed12d03bf97e89e590614e11c6d47
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275160
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
Just let cloudfront compress the assets for us. It can handle the brotli/gzip
switching natively now.
Change-Id: I3416d428403f0bfbec5a575ff1c095d2342a5e4c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247818
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
refs VICE-2034
i notice that some specs use
have_receieved instead of increments
from the looks of it, these tests
look hardier than simply using receive
p.s. I really wanted to add the new cop
to this commit, but adding a cop is a big effort
in and of itself. it will be tackled in
the next commit after this one is merged
Change-Id: If4a6bde01017abb5ab4c16cdd5d056e4caf1788a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273314
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Drake Harper <drake.harper@instructure.com>
Reviewed-by: Brian Watson <bwatson@instructure.com>
QA-Review: Drake Harper <drake.harper@instructure.com>
Product-Review: Drake Harper <drake.harper@instructure.com>
closes OUT-4695
flag=none
test plan:
- ensure live events are running locally (see live_events.md)
- verify that when making changes (including soft deleting) or
creating an outcome friendly description that live events are emitted
Change-Id: I98ae5aeda1e16e1816f7d63a6a82f777a2766ae0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272071
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Augusto Callejas <acallejas@instructure.com>
Creating a Kinesis client with no creds causes the AWS gem to reach out
to an AWS-internal IP (169.254.169.254). There is a connect timeout of 1
second but this is disabled in
config/initializers/no_timeouts_debugging.rb. As a result, creating a
Kinesis client can indefinitely hang, making Canvas completely unusable.
The issue is made worse by recent changes which turned Live Events on
with such invalid config for anyone who used config from
dynamic_settings.yml, whereas previously, Live Events had to be manually
turned on via a plugin.
This change prevents invalid configs from reaching the Kinesis client;
except in prod in case anyone is actually using the feature whereby
AWS looks up creds on the internal IP (and to avoid any possible
slowdowns by checking for the settings repeatedly).
Additionally I made NoRaiseTimeoutsWhileDebugging log a message when a
timeout happens, which would have greatly helped with debugging this
issue.
refs INTEROP-7016
flag=none
Test plan:
- Have config in dynamic_settings.yml like the old
dynamic_settings.yml.example, with live-events config but no creds,
e.g.:
live-events:
aws_endpoint: http://kinesis.canvaslms.docker
kinesis_stream_name: live-events
- From a rails dev console run LiveEvents::Client.config. It should
return nil.
- Run Canvas and make sure you can login, view a course, etc. -- just
try anything that emits a live event (almost anything)
- Add aws_access_key_id and aws_secret_access_key_dec from from
dynamic_settings.yml.example into your dynamic_settings.yml
- From a rails dev console run LiveEvents::Client.config. It should
return the config.
- From a rails dev console run `LiveEvents::Client.new`. It should
immediately produce a client.
- Restart Canvas and make sure you can view a course, etc.
Change-Id: I9a325b7f30c8e0203c2903a25a1f0139776b3f1f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271907
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
* and change singleton key
why:
* catalog is getting much fewer course_completed live events than they
used to
* live events log diving reveals the events _are_ getting sent, but as
course_progress instead, even though the API shows that the course
has been completed
* the on_conflict option defaults to :use_earliest, which doesn't
completely explain these discrepancies, but certainly isn't helping,
since the docs for inst-jobs say to use on_conflict: :overwrite for
debouncing.
* also, uncovered that sometimes if multiple modules are completed in
the 2 min debounce period, multiple jobs will be enqueued since the key
for the singleton strand is the ContextModuleProgression id. It would
be better if the key used the course and user, so that only one job and
only one event get sent at a time.
* this isn't a final solution but is a helpful first step
closes INTEROP-6924
flag=none
test plan:
* in `live_events_callbacks.rb:159`, change the delay_if_production to
just delay, for testing
* follow the directions in `doc/live_events.md` to set up live events
piped to the canvas docker kinesis stream, and tail the stream to see
live events flowing (reminder: after configuring the Plugin you *must*
restart canvas).
* have a course with a student and 4 modules, each with one
assignment in them
* as the student, submit the assignment for the first module
* in a rails console,
`Delayed::Job.where(tag: "CourseProgress#dispatch_live_event")` should
show you the jobs enqueued for the student
* there should only be one, and the strand should include the global ids
for the course and user
* the run_at time should be for two minutes from the created_at time
* in the browser, submit the assignment for the second module
* the run_at time should update to two minutes from now
* wait two minutes and let the job run
* in the stream logs, you should see a course_progress event
* in the browser, submit the assignments for the last two modules
* another job should be enqueued, wait two minutes and let it run
* in the stream logs, you should see a course_completed event, and no
more course_progress events (besides the first)
Change-Id: Ib6860a185031a9b9f0375347e079a0501f3b571c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269276
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
fixes INTEROP-6911
also fixes some specs in jwts_controller_spec -- the intention was to
use a services JWT to try to generate a services JWT and see it fail.
it did fail, but for the wrong reason -- the spec didn't actually pass a
services JWT in the auth header, it passed something unrecognizable.
the fix here is to make it a proper JWT, and add a more specific
assertion so we're confident that we're exercising the right code path.
test plan:
- see test plan for commit c44b1844f7 for how to generate RSA keypairs
and configure InstID to use them
- get an API access token by following
https://canvas.instructure.com/doc/api/file.oauth.html#manual-token-generation
- use that token to generate an unencrypted InstID token like:
$ curl 'http://localhost:3000/api/v1/inst_ids?unencrypted=1' \
-X POST \
-H 'Authorization: Bearer <access_token>'
- now use the returned token to hit some other endpoint, e.g.:
$ curl http://localhost:3000/api/v1/users/self \
-X GET \
-H 'Authorization: Bearer <InstID_token>'
Change-Id: Ie6c01dd12d98f68c138da1960b87f91e42b8b04d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269345
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
fixes INTEROP-6913, INTEROP-6892, INTEROP-6893, INTEROP-6920
flag = none
This commit introduces the InstID token, a signed and encrypted JWT (aka
JWE) that will soon be usable for Canvas API access (that's "part 2").
If the InstID class is configured with a private signing key and public
encryption key, it will be able to produce encrypted JWTs and validate
and deserialize decrypted JWTs. If it is configured with only a public
signing key, it cannot produce tokens but it can still validate and
deserialize decrypted ones. Therefore this class can be used by the
identity provider (currently Canvas) to produce tokens, but also by any
services that want to use InstID tokens for authentication.
test plan:
1) generate two RSA keypairs. one way to generate a keypair is from a
rails console:
> keypair = Canvas::Security::RSAKeyPair.new
> puts keypair.private_key.to_s
> puts keypair.public_key.to_s
2) choose which one is for signing and which is for encryption, then add
the private signing key and the public encryption key to your rails
credentials:
- run `bin/rails credentials:edit`
- add an entry like the following, and then save and close your
editor:
```
inst_id:
encryption_key: |
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvY1EMlGm1daM87ejGuFX
<...snip...>
/wIDAQAB
-----END PUBLIC KEY-----
signing_key: |
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAnDwED/QOB0f0H6TOZqLmjaPqA7m8c40NDXkAa6u5cK8zCbk3
<...snip...>
QhjPgifBwTrzj21484CfiPfy5oe756Exerj8PIlRrE/hxWRSDwBIOg==
-----END RSA PRIVATE KEY-----
```
3) open a rails console and do:
> id = InstID.for_user('user-uuid')
> id.to_token # make sure this doesn't blow up
> token = id.to_unencrypted_token
> decoded_id = InstID.from_token(token)
> id.jwt_payload == decoded_id.jwt_payload # => true
TODO in followup commits:
- make canvas accept InstID tokens for auth
Change-Id: Ie550c17507c26f9944bd62a747a6a63161e8e770
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268872
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
closes FOO-1766
flag = none
[fsc-max-nodes=18]
[fsc-timeout=30]
Test Plan:
- Run the migration and make sure there are no errors
- Some things to check:
* How it acts as a teacher, student, and public user
in course files/folders and personal files/folders
with the various settings above toggled to different states
* How it acts as a teacher, student, and public user
in discussions, modules, content migrations/import/exports
(RCE should behave similarly throughout the site)
* Should only be able to upload or add folders if the
Course Files - add permission is enabled for the user's role
* Should only be able to manage file access, usage rights, move,
or rename course files/folders if the Course Files -
edit permission is enabled for the user's role
• Check Toolbar header at the top of Course files
• Check Cog (hamburger menu) to the right of each file/folder
• Check Usage Rights Indicator under usage rights column
that can be found in course and group file pages. This can
be enabled under course settings if not available
* Should only be able to delete course files/folders if the
Course Files - delete permission is enabled for the user's role
* Any given user/role should have full access to their respective
personal files/folders regardless of granted permissions. The
same also applies to a group context with some caveats
• Should not be able to modify file access in a group context
• Should not be able to modify usage rights in personal files
* A student enrollment not granted any file permissions (the default)
should only be able to _view_ and _download_ files unless granted
additional access from an authorizing role
* REST API works as expected
* UI works as expected with no additional javascript errors
Change-Id: I5e7f717494d658e6c8ec9be8a8039015afcebc63
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262775
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
refs FOO-2036
basically, refresh tokens took over the meaning of expires_at to mean
when you needed to refresh. another column is needed to mean you
need to refresh vs. the token is expired, period. I opted to add
a new column for permanent_expires_at instead of needs_refresh_at
because the datafixup is less racey this way.
Change-Id: Ia11a2e862e540f211d628aa39c05bcb6930647ac
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266765
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Change-Id: I5c1a9f260afc56887881f98cdab08ad3a25c45f3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263980
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Since we cache by full key anyway, there is no good reason
to convert it into a recursive hash only to convert it right back
Change-Id: Ifb356e38a2ac5e6e0477ebc5913e0d9128622489
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263932
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
refs FOO-1853
flag = none
test plan:
• tests pass
• spec is no longer flaky (verify via splunk)
Change-Id: I0907bb81c6d053caaccc88c1385b2d4f6c17f81a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262581
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
In this case we don't care what arguments
ConfigFile#load is receiving, we just care that
the proper response is getting set for the method stub
Using .with can cause fragility if the expected args
don't match what the method stub is actually being
called with.
fixes FOO-1853
flag = none
test plan:
• tests pass
• spec is no longer flaky (verify via splunk)
Change-Id: I8800748d175aeadae6dccbcac011418679adde4e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262408
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
refs QUIZ-8160
flag=none
test plan:
- test with quiz_lti patch /quiz_lti/+/261845
Change-Id: Ib8e1f8f2f5449b8ebbadea6067f8aaeef925c8f9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261846
Reviewed-by: James Logan <james.logan@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
Product-Review: Susan Sorensen <susan.sorensen@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs OUT-4247
Test plan:
- ensure results from aligned rubrics and from quizzes
are visible in the learning mastery gradebook
Change-Id: Ie9731f2c702f91b3da2ce4d8ae43f7045ee5509e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260613
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
the ContextModuleProgression may be saved more than once while
requirements are evaluated, and these saves may generate spurious
course completion live events based on incomplete data. evaluating
an item requirement may also trigger delayed re-evaluation of other
requirements.
give these changes time to settle before sending any course progress
live events. queue a singleton job to do so, so multiple changes
in close succession do not result in multiple live events. the
delay is configurable as the "course_progress_live_event_delay_seconds"
Setting (default 120, two minutes).
test plan:
- enable live events in a production-like environment
- have a set of modules with requirements
- as a student, complete some of them in rapid succession
- wait two minutes, and a course progress live event should be
created and up-to-date when received (counting the right
number of completed requirements)
- as the student, finish the requirements
- two minutes later, a course completion live event should be
received
alternatively you could modify the Setting and customize the
expected delay
closes LS-1967
Change-Id: I5c5ab01e75c48cc9916d9299ee1be55d360d1ee7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260416
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
refs FOO-1648
flag=none
move general module code to "gems"
along with specs.
Leave shim in canvas to avoid breaking
things while callsites get changed.
change some limited callsites
from Canvas::Errors to CanvasErrors
that were doing spec stubbing that required
at least one namespace change anyway
generate readme describing common use
cases.
TEST PLAN:
1) we still send stuff to sentry
Change-Id: I0111afc21e9e8c6604370885b4c750186b96175f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260035
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1648
flag=none
[fsc-propagate=false]
move general module code to "gems"
along with specs.
Leave shim in canvas to avoid breaking
things while callsites get changed.
change some limited callsites
from Canvas::Security to CanvasSecurity
that were doing spec stubbing that required
at least one namespace change anyway
generate readme describing common use
cases.
TEST PLAN:
1) security operations continue unimpeded.
Change-Id: Ia2d102d5038e2f5d0bb24201e38894e12b73063e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259540
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1636
flag=none
[fsc-propagate=false]
- add gem initializer to canvas for canvas_cache
- move Canvas::Redis into canvas_cache
- alias module in canvas
- expand dependencies of canvas_cache
- add redis dependency to vendored gem environment
- extract RedisConnections to deal with disconnect/clear_idle
TEST PLAN:
1) continue to cache things in redis
2) caching behavior with canvas-extensions
should not change
Change-Id: I96383eced80d670e14e535993f0c0608ab5f6520
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259434
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1615
flag=none
first step of verifying modularization path for
chunks of behavior inside canvas.
take a dependency on config_file to minimize
unnecessary injection
move initialization of dynamic_settings to one place.
leave pointer in previous initializer file.
TEST PLAN:
1) nothing changes at runtime
2) dynamic settings specs can be run
quickly on their own
Change-Id: I1bfbd5b8dfedb9835953d9fc663259c17b76ce7f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258762
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1648
flag=none
These login audits are a group of functionality
very specific to the pseudonym login case,
and they know a few things about the pseudonym model.
Moving it into it's own module makes sense from
and organizational standpoint, but also makes it
more reasonable to extract the rest of canvas_security
into a gem for other gems/engines to depend on.
TEST PLAN:
1) the pseudonym login process remains unchanged
2) specs pass
Change-Id: I8f689c9255c33f0ac4d7947e3b0a0592d7900463
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259557
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
refs FOO-1648
flag=none
Canvas::Security depended on
ServicesJwt for secret loading.
ServicesJwt depended on Canvas::Security
for decoding and decryption
pushing the secret loading into the
security module breaks the circle
so we can extract canvas_security
without having to bring everything
at once.
TEST PLAN:
1) JWTs continue to get decrypted as they have
2) specs pass
Change-Id: I08c42b9727bee3660998cc2b0d9139beb496e244
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259550
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1648
This method doesn't belong in core
security because of all it's dependencies
on models and PluginSettings. These are circular
because models depend on Canvas::Security.
We need to move it so we can extract the rest
of canvas_security safely.
TEST PLAN:
1) mass re-encryption works as expected
2) specs pass
Change-Id: I1e28d83b09e36e9d72408a7dbd7276329162434b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259545
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs FOO-1636
flag=none
this helps ease the migration of caching
functionality into the canvas_cache
gem by unwinding circular references between
these modules.
Canvas now invokes Canvas::Redis, but
Canvas::Redis no longer invokes methods
on Canvas.
TEST PLAN
1) caching continues to work the same in operational environments
Change-Id: I99a55a63def4fd01e6195f26ec69ae8f62d76314
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259424
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
it's only public sometimes because of how it's been extended
Change-Id: I64793fe54a6a98e7b509f968bf53aa6ba349f6fd
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258931
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
closes FOO-1552
flag = none
test plan:
• permissions around course_sections function as they did before,
but now they are granularized and permanent
Change-Id: I5918a4c6ccce2ccaaf87d4b92772c31ad3e6280c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257619
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
closes FOO-1501
refs FOO-130
flag = granular_permissions_course_files
[fsc-max-nodes=18]
Test Plan:
• see test plan outlined in base commit: g/253777
Change-Id: I33984062fd236348d39262395e5f51335e327ed9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256914
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Note: we'll want to re-run the data fix-up when we're ready to turn
on the feature flag permanently; in hopes to capture any differences
made to course files permissions between now and then.
Modified the files_controller quota and api_quota permission checks
to make them more lenient in regards to accepting any or all of the
files permissions role overrides. This allows legacy grouping and
new granularized files permissions to live in harmony and be modified
without causing unauthorized errors on the quota resource.
This commit will cover the backend permissions required to granularize
files / folders permission calls, however there will be a follow-up
ps to clean up the course file page to hide elements the user might
not be authorized to use.
closes FOO-130
refs FOO-1501
flag = granular_permissions_course_files
[fsc-max-nodes=18]
[fsc-timeout=30]
Test Plan:
- Run the migration and make sure there are no errors
- With the granular_permissions_course_files FF turned off,
course sections and REST API should work the same with this patch
set checked out as it does in beta/production
- Some things to check:
* How it acts as a teacher, student, and public user
in course files/folders and personal files/folders
with the various settings above toggled to different states
* How it acts as a teacher, student, and public user
in discussions, modules, content migrations/import/exports
(RCE should behave similarly throughout the site)
- With the granular_permissions_course_files feature flag turned on
course files/folders and REST API should work as expected. The same
list checked above should be done so again, but this time:
* Should only be able to upload or add folders if the
Course Files - add permission is enabled for the user's role
* Should only be able to manage file access, usage rights, move,
or rename course files/folders if the Course Files -
edit permission is enabled for the user's role
• Check Toolbar header at the top of Course files
• Check Cog (hamburger menu) to the right of each file/folder
• Check Usage Rights Indicator under usage rights column
that can be found in course and group file pages. This can
be enabled under course settings if not available
* Should only be able to delete course files/folders if the
Course Files - delete permission is enabled for the user's role
* Any given user/role should have full access to their respective
personal files/folders regardless of granted permissions. The
same also applies to a group context with some caveats
• Should not be able to modify file access in a group context
• Should not be able to modify usage rights in personal files
* A student enrollment not granted any file permissions (the default)
should only be able to _view_ and _download_ files unless granted
additional access from an authorizing role
* REST API works as expected
* UI works as expected with no additional javascript errors
Change-Id: Ieb2d10915c274959e8da4c623f7aba11d3540c2b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253777
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
refs FOO-1449
flag=none
TEST PLAN:
1) update consul state to have a current key
and previous key
2) tokens are still accepted when signed by old key
3) remove old key from consul
4) tokens signed with old key no longer accepted
Change-Id: Ie31b7ad1cdab1ab779210c75524ce3559423a44c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256346
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1397
flag=none
TEST PLAN:
1) run a big web application with many reads
2) requests to consul read the right things
even shortly after forking on a busy box.
Change-Id: Ie416b6a145a00b4cc972bdf4e3c3e4c1db31e684
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256330
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
* note that these are lti ids, not canvas ids
* source course is present if it's a course copy, otherwise it's nil
* destination course is always the context of the migration
closes INTEROP-6373
flag=none
test plan:
* specs
* if desired, send a content_migration_completed and note the presence
of those fields
Change-Id: Ibc1816ee715dc27420d1d35522924f3dc33c3152
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254629
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
fixes INTEROP-6354
flags=none
Test plan
- Set up a tool proxy with live events
- Ensure the live events that get sent
have the appropriate associated integration id
- Ensure that the subscriptions are getting created
with the appropriate associated integration id
- Ensure the data fixup corrects the subscriptions
Change-Id: Icdd82b42ead45eadc5b9456a66c52eedb23d6be5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253160
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
closes FOO-1272
flag=none
Use a specific error message for missing
queued messages that doesn't require
a sentry error for each.
Also introduces a pattern for
"Retriable" job errors and the
accompanying error-level handling
TEST PLAN:
1) delay the "deliver" ing of
a message that does not exist.
2) you should get a log message,
but no sentry error
Change-Id: Ib6d8f5ae499b2e1ecde1eb74397d77630c0f3e6b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254639
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
fixes FOO-1163
test plan:
* have a user with pseudonyms on two accounts with the same local id, on
different shards
* go to the user's page (/users/:id)
* it should correctly name both accounts
Change-Id: Ib5dfb77163a013deb0029dc311b26b1b9424f76c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252039
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-1125
flag=non
TEST PLAN:
1) stats for things like ImperiumTimeouts should still
end up in datadog
2) sentry errors for the target error types should disappear
Change-Id: I6e97c04e3f6fcc3545b10418511934c89f20a419
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251536
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
closes FOO-1120
refs FOO-1125
flag=none
introduce canvas-errors logging callback
move error logging from application_controller
to Canvas::Errors callback
let all canvas::errors callbacks handle the new options hash
don't send errors less than ":error" level to sentry or
error reports.
change error stats to be grouped by "level", and use
error name as a tag.
TEST PLAN:
1) regular errors should still hit all the callbacks
2) errors declared as ":warn" or ":info" levels should
not make it to sentry (like SAML MissingMessage errors)
3) local errors during development should still get logged out
to development.log file
Change-Id: Ibe1401c0f5ad8f6d697c64f6a34153aaea048281
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251125
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
test plan:
* account feature flag cache register checks should
be made against multicache redis, just as with g/246223
closes #LS-1404
Change-Id: I28172e9c7bf539c98d63b323f911b9e6bdcaf909
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246243
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
closes OUT-3999
flag=none
test-plan:
- ensure live events are running locally (see live_events.md)
- verify that when making changes (including soft deleting) or
creating an outcome calculation method that live events are emitted
Change-Id: I715305dc3cd8dc1cab127033d1feb92acd28f948
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250457
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Manoel Quirino <manoel.quirino@instructure.com>
Product-Review: Jody Sailor
"Somehow, the rotate_keys job returned" --
on Oct 1, both Lti::KeyStorage.rotate_keys and
Canvas::Oauth::KeyStorage.rotate_keys ran multiple times within seconds,
causing the keys to get rotated twice. This commit prevents us from
rotating keys if we just did a few seconds/minutes ago.
flag=none
closes INTEROP-6248
Test plan:
- in a rails console, run Lti::KeyStorage.retrieve_keys to see current
keys. (Local Canvas didn't seem to be saving DyanmicSettings for me so
rotating keys didn't have any effect on the keys at keys in
/api/lti/security/jwks)
- kick off rotation of keys: Lti::KeyStorage.rotate_keys
- check keys rotated
- kick off rotation of keys again
- check keys not rotated
- change min_rotation_period to something smaller like 2 minutes. You
can do that with:
class << Lti::KeyStorage; def min_rotation_period; 2.minutes; end; end
Wait that amount of time, kick off rotation of keys again, and check
that the keys rotated.
- run Canvas::Oauth::KeyStorage.rotate_keys and
Canvas::Oauth::KeyStorage.rotate_keys to make sure the min rotation
period for each set of keys is completely separated. Note that these
seem to start out empty so you will have to rotate them once to generate
them.
Change-Id: I7d58909e2eba7ccb2cd4222e60edf3f437547a6e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/250155
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Normally submissions are created in raw SQL so the after_create hook is
not fired but there are some cases where this is not the case: in these
weird edge cases, submission_created is also fired when we create
Submission records with ActiveRecord so the after_create hooks fire. We
don't understand exactly what they are but they are a very small
proportion (about 0.15% of total submission_created events) and no one
cares about them.
This also adds "workflow_state" to the event body, and DRYs up some
specs.
closes INTEROP-6224
flag=none
Test plan:
- Run canvas with the environment variable STUB_LIVE_EVENTS_KINESIS=1
e.g.:
dcr -e STUB_LIVE_EVENTS_KINESIS=1 -u 0 web \
bundle exec rails server --binding='0.0.0.0' -p 80
- create assignments and make submissions for students in the following
ways, making sure the submission_created event is emitted (it will
be in the Canvas STDOUT, you might have to search for it) only when the
student (or API call in some cases) submits or resubmits:
- Student submits through UI.
- Student re-submits through UI.
- Done through API (?) Something like
/api/v1/courses/123/assignments/12345/submissions/1234 ?
- Grade passback (?)
- AGS (?)
- Make sure a submission_created event has the "workflow_state" in it.
- In a console, create a Submission with Submission.create! in a
unt push gerrit
submitted state and make sure the event does not fire.
- In a console, create a Submission with Submission.create! in a
submitted state and make sure the event does fire (you must provide
workflow_state: submitted, and a submission_type such as
online_text_entry)
Change-Id: I92b1c595282bf498557b7e52660547cc624795ac
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249769
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Product-Review: Oxana Jurosevic
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Fixes INTEROP-6252
flag=none
Test Plan:
- Install multiple plagiarism detection tools
in the same account with _different_ resource
type codes. Each tool should also have a different
endpoint
- Associate an assignment with one of the tools
- Submit and verify an event is sent to the tool
with the matching resource type code
- Verify events were not sent to the other tools
Change-Id: I5892451c2ac3af64254881319d6f1143703a5cb6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249776
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
fixes INTEROP-6250
flag=none
Test plan
- Set up tools with varying endpoints and verify
we're sending the "closest/latest" endpoint ids
from among a plethora of similar tools on courses/
sub-accounts/accounts
- Verify the migration does the same thing
Change-Id: Iac761e94cfeee01588cae5cd8b4797faa1bcb643
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249736
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
so we know where they came from
Change-Id: I41d58d73c99d1187064f7d83821f361d3d73791f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249256
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-1032
we don't want to block local redis for very long,
it's not like the much more variable traffic
experienced by remote redis rings. Just give it
a little time to recover, and let us use a different
setting.
TEST PLAN:
1) make your local redis fail to connect
2) after the rails process acknowledges and blocks that host,
restore local redis.
3) within a few seconds you should be successfuly serving
queries from the local cache again.
Change-Id: Ic8bf976418198e9b5d6fa02a74b67edc60012a46
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249274
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1032
also log even expected redis failures
so that it's possible to pin down why a given request
failed after a botched redis query.
TEST PLAN:
1) make your redis client irredemably faulty with
a caught error, like InheritedError
2) try to ask MultiCache for results with an "mget"
3) you should get a [] as the default return value
Change-Id: Ia8ede817d1180e6c0a42dfbe69bfbcb5040c4b4d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249255
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Skipped spec has had the success rate dip into the 30% range, well
below the acceptable flakiness of a spec.
flag = none
Change-Id: Ib41a6517e39ceb650e44073d03f5481be7825af4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249241
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Some endpoints (such as GraphQL and the LTI AGS scores service) appear
to not set `@context` in ApplicationController so the LiveEvents context
(which becomes "attributes"/"metadata" in live events) does not have a
context_id / context_type. This is a problem for some downstream plagiarism
platform consumes of the submissions_* events (particularly
submission_created).
flag=none
closes INTEROP-6174
Test plan:
- submit an assignment thru the grade_passback, AGS "Scores", or graphql
and check that the live event has a context_id.
Change-Id: Ife5b213436de63ebcef179b0a9ba41a7b088d182
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/249001
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Oxana Jurosevic
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
usage rights were only never implmenented in canvas for users. Don't
show the UI in RCE's file upload modal in a user context.
closes LS-1457
flag=rce_enhancements
test plan:
- as a teacher, create an assignment
- open Documents > Upload Documemnt (or Image > Upload Image)
> expect the Computer tab to have the Usage Rights UI.
- pick one and submit
> expect the file to have the copyright symbol in the course
files page
- as a student, submit the above assignment
- in the RCE, open Documents > Upload Docuemnt
> expect no Usage Rights UI to be displayed
- create a group in a course with usage rights required
- as a teacher on a group homepage, create an Announcement or a Page
- open Documents > Upload Document
> expect to have the Usage Rights UI
- pick one and submit
> expect to see the file have the copyright symbol in the group
files page
Change-Id: Ib340c180885affda24af8cd809b06a8d565e0f34
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248488
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Ed Schiebel <eschiebel@instructure.com>
closes FOO-1013
TEST PLAN:
1) send a sighup to a box
2) only ONE flushdb gets sent to local redis
Change-Id: I5af89014d21b79382cada9aa7b32eaa4e4e088b2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248800
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-779
TEST PLAN:
1) make many threads or processes try to load the same vault config
2) only one should actually call vault
3) they all should get a response
Change-Id: Ic5b778ab994bd964d930fd40fee936f42a4fd91d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248155
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
FIXES FOO-1011
FIXES FOO-1012
Also write the trees of dynamic settings
stuff atomically so that clears don't
result in a partial config hash in the cache
TEST PLAN:
1) store things in the dynamic settings cache
2) explicitly clear the cache
3) write a "true" value to the tree-level key
4) fetch the subkey
5) you should still get a valid value back from consul/file
Change-Id: I0f8bf524a6c03a2e6a6bba7fed3c199e6b179d93
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248752
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-965
raise a real error if we get a response
other than a 200 or 404, only 404 is a good
reason for putting a nil value into the cache.
TEST PLAN:
1) override imperium to return error responses
2) run dynamic settings lookup
3) nil is not in cache, error reported
Change-Id: Iba384c122e674864fbcaf49c3159ab8aca3f0833
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247948
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-965
TEST PLAN:
1) throw an error in a request
2) catch the error and capture with canvas errors,
then continue
3) you should get request and session info on
sentry context automatically
Change-Id: Iadacc5337eee5298c9190b34a976a482e6407572
Change-Id: If654cdd76f5b39218bd633b66d3c073c083185af
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247924
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
sharding isn't necessarily set up if this
fires during boot sequence
Change-Id: I2d5448184f141716bd38eb29beef33fe155abdb7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247927
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
fixes INTEROP-6125
flag=none
Test plan
- Set up an assignment with the plagiarism tool
- Test to make sure the live event sends with
the associated integration id from the ACTL
- Uninstall the tool and make sure it does
not send the associated integration id
- Play with different locations where the
tool is installed and make sure it works
as expected
Change-Id: I1fec2ee4744850cd7a0285fb0087a465b3448e46
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246294
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
refs FOO-779
TEST PLAN:
this doesn't get used yet,
but if you want you can grab
any cred path your vault client
is allowed to assume and try to pull
it through this object and talk to
an AWS service the assumed role is
permissioned for
Change-Id: I85faf802539feae6e708dfa8bd07d1f52346df1d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246255
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
closes INTEROP-6016
flag=none
Test plan
- Have a submission that isn't graded
- Comment and/or grade the submission
- Note that a live event gets sent with
a new posted_at date
Change-Id: I63175965044350187af2b9c4857fb25b0ebc2e68
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244908
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
closes OUT-3867
flag=none
test-plan:
- ensure live events are running locally (see live_events.md)
- verify that when making changes (including soft deleting) or
creating an outcome proficiency that live events are emitted
- verify that ratings information is included in the proficiency payload
(cherry picked from commit 0f0edc8cbe)
Change-Id: Iee0a98666354fc6e375757d63a1bbf22cdff973e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245081
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Brewer-Davis <mbd@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
refs SAS-1540
* adds an audience setting to developer keys, so a key can be set to
target external audiences with its credentials grants
* when a key with an external audience grants credentials, the token is
signed with an asymmetric key instead of the internal symmetric key
* external audiences can retrieve the corresponding public keys from
/login/oauth2/jwks
* credentials issued by developer keys with an account id include the
account's guid in a custom claim
includes a refactor of key storage and rotation in consul, which had
already been done for LTI. but it wasn't really a feature of lti, just
something used by LTI, and we needed the same for key management for
this. moved it to be part of Canvas::Security
Change-Id: Ie5c0fcee6fc21687f31c109389a3bcc1ed349c5d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243606
QA-Review: Jonathan Featherstone <jfeatherstone@instructure.com>
Reviewed-by: Jonathan Featherstone <jfeatherstone@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
This reverts commit 0f0edc8cbe.
Reason for revert: merged prematurely, depends on code in g/244166
Change-Id: I49b623197354bf5c39cb3f6829b2dea72118e890
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244037
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Pat Renner <prenner@instructure.com>
Product-Review: Pat Renner <prenner@instructure.com>
closes OUT-3867
flag=none
test-plan:
- ensure live events are running locally (see live_events.md)
- verify that when making changes (including soft deleting) or
creating an outcome proficiency that live events are emitted
- verify that ratings information is included in the proficiency payload
Change-Id: I0e6603896b6a08b7a6ea1050c742e5ff3a9b3b2f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244793
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Pat Renner <prenner@instructure.com>
refs FOO-739
TEST PLAN:
1) disable request throttling
2) run some requests that are expensive
3) costs should still be tracked in headers
Change-Id: Iec1a60f797451c789c3cdb79682757b8c354005f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243569
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
test plan:
* with PG <10, nothing should change/break (migrations shouldn't even try to run)
* with pg >=10, run migrations
* it should create collations, and recreate a couple indexes
* going to the users page should not be broken
* specs ensure that the sorting is to our liking
Change-Id: I3eaf42dd2207d733c164c12ef2a43a1c1b417ff2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241190
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
closes PLAT-5766
flag=none
Test plan
- Planning to test this on beta once it's there
as that will be easier than trying to test it
locally
- We'll want to install a fresh Plagiarism tool
at an account
- Verify that the plagiarism subscription IS receiving
events for assignments that are linked to the tool
- Verify that the plagiarism subscription IS NOT receiving
events for assignments that are not linked to the
tool
- Verify that regular subscriptions are still receiving
events for both linked and non-linked assignments
Change-Id: I59a1f5f6d1c061dfcd50c1efa2788173875d2231
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/240369
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Guymon <mguymon@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
flag=none
refs PLAT-5791
Test plan:
- make sure you have a course with a sis_source_id
- tail live events kinesis stream (See doc/live_events.md)
- trigger a course_completed or course_progress event
- check it has the account ID and sis_source_id
Change-Id: Ic4e529ee7b3eeddce68e028fcb81c5bdbbafb96a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/239269
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Oxana Jurosevic
Ahmad Amireh