Commit Graph

974 Commits

Author SHA1 Message Date
Ethan Vizitei 8aafc7d464 unconfuse OAuth::Helper references
refs FOO-2529
flag=none

TEST PLAN:
  1) sentry error goes away

Change-Id: I32305a316832fa21bd1edb8550cba309a3222c35
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276823
Reviewed-by: Ben Rinaca <brinaca@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-10-26 16:18:22 +00:00
Aaron Ogata 5ec29f69d0 only spawn threads for files that will actually upload
refs DE-859

Change-Id: I625a36b823a3ed7d879e0e1da266c235c74a1f0a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276752
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-10-25 18:03:14 +00:00
Aaron Ogata b5e77f261f add verbose logging option to s3 uploader
refs DE-859

Change-Id: I4e4615cf57ec47afc0a405848eec6302a11c9702
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276609
Reviewed-by: Alex Slaughter <aslaughter@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
2021-10-22 19:51:15 +00:00
Jacob Burroughs b329523126 Migrate twilio to use rails creds
fixes FOO-1496

Change-Id: I26e329e09022aecca3f575667fe5ff8c377eb24d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276546
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2021-10-22 14:51:17 +00:00
Jacob Burroughs e4e7aa4814 Expose vault config
Change-Id: I8f4518995f617e5baf8f878967e58ce865dce850
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276501
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
2021-10-21 15:51:47 +00:00
Jeremy Stanley cabece352d log job ids in exports and imports
if something goes wrong, but not so wrong that the entire export
or import fails (so there is no captured exception), it's
essentially impossible to review logs of the migration. if we
store the job id, we can provide it to the log-downloader script.

test plan:
 1. a. perform a common cartridge export
    b. in the console, examine ContentExport.last and ensure
       settings[:job_id] is set
    c. look at log/delayed_job.log and verify the log lines
       for the recent export include that id
 2. a. perform a course copy
    b. in the console, examine ContentMigration.last and ensure
       migration_settings[:job_ids] contains a job id
 3. a. perform a common cartridge import, choosing to import everything
    b. same as 2(b)
 4. a. perform a common cartridge import, choosing to import selectively
    b. make some kind of selection and complete the import
    c. ensure migration_settings[:job_ids] contains two job ids
       (one for each stage of the migration)

flag = none
closes LS-2730

Change-Id: I5d4028f4384111a06fa10d6327dce99c487fb9d6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275929
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2021-10-15 16:11:28 +00:00
Alex Slaughter 30f0a02445 Add $Canvas.assignment.description to LTI launch params
Add support for LTI tools to consume the assignment description
set in canvas.

Test Plan:
 - Specs pass

flag = none

Change-Id: I339d80f91fbe2bf2e9c855f227b9ee015f1bb835
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275548
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Alex Slaughter <aslaughter@instructure.com>
Product-Review: Alex Slaughter <aslaughter@instructure.com>
2021-10-15 06:08:05 +00:00
Ethan Vizitei 8b9c0d126d wrap core_ext module in Canvas for zeitwerk
refs https://instructure.atlassian.net/browse/FOO-2476
flag=none

TEST PLAN:
  1) specs pass

Change-Id: Iaa6a7150eb1eb0bd2bc778a5ef2df540eefde500
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275962
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
2021-10-14 22:39:18 +00:00
Ethan Vizitei 015d7abc62 inflect OAuth for zeitwerk
refs FOO-2476
flag=none

TEST PLAN:
  1) specs pass

Change-Id: I9f73e0021f5efec2be3c2f419e758baf00e6914f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275642
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-10-13 16:32:05 +00:00
Ethan Vizitei d08c6f721a rename core_ext/oauth2 for zeitwerk compatibility
refs FOO-2476
flag=none

TEST PLAN:
  1) specs pass

Change-Id: I26892b88a145b7ba95435be125f4ce2387cfc85c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275631
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-10-11 21:31:32 +00:00
Ethan Vizitei b1b4b53e69 wrap default plugins registration in a module and function
closes FOO-2445
flag=none

change application.rb to invoke the plugin
apply function rather than just requiring the file.

TEST PLAN:
  1) specs still pass
  2) plugins are still loaded on boot (selenium spec confirm this)

Change-Id: Ia6ada545c99cc5e33f6ad086a84e6e4bbe859833
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275407
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-10-07 19:22:06 +00:00
Ethan Vizitei 1ce41c20fe rename CCWorker so module path can match file path
closes FOO-2436
flag=none

TEST PLAN:
  1) all specs still pass

Change-Id: I958149a8537e704b5144adb7121c0318bcb5c823
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275403
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-10-07 19:21:40 +00:00
Ethan Vizitei eab0f468e0 reset message bus settings to avoid state pollution from specs
refs FOO-2423
flag=none

TEST PLAN:
  1) run pulsar specs lots of times with many seeds
  2) they don't fail.

Change-Id: Ide7fbaebee5ddac37bf2db03d8699f32d1b57d56
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275315
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-10-06 17:13:21 +00:00
Jacob Burroughs 214014049f Add asymmetric encryption for service tokens
refs FOO-2410

test plan:
- in dynamic_settings.yml, add the following block:
```
store:
    canvas:
      services-jwt:
        # these are all the same JWK but with different kid
        # to generate a new key, run the following in a Canvas console:
        #
        # key = OpenSSL::PKey::RSA.generate(2048)
        # key.public_key.to_jwk(kid: Time.now.utc.iso8601).to_json
        jwk-past.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-05-18T22:33:20Z_a\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
        jwk-present.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-06-18T22:33:20Z_b\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
        jwk-future.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-07-18T22:33:20Z_c\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
```
- Ensure /internal/services/jwks loads correctly
- In console, ensure `CanvasSecurity::ServicesJwt.decrypt(Base64.decode64(CanvasSecurity::ServicesJwt.for_user('localhost', User.first)))`
and `CanvasSecurity::ServicesJwt.decrypt(Base64.decode64(CanvasSecurity::ServicesJwt.for_user('localhost', User.first, symmetric: true)))`
both work and produce sensible looking output

Change-Id: I13c6c35cc92ed12d03bf97e89e590614e11c6d47
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275160
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
2021-10-06 15:11:06 +00:00
Cody Cutrer 274ea77dff RuboCop: Style/Alias
(auto-correct)

also tweak a couple other cops

Change-Id: I22c7b178d468b5107251437434fcd3d0a4b572f1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275260
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-10-05 21:12:23 +00:00
Cody Cutrer 3ef09eed34 allow configuring token expiration for mobile apps
fixes FOO-2336

Change-Id: Ic7f555ae4d85b8b79c7114f43495a638ca2dead2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273587
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-10-04 20:10:39 +00:00
Cody Cutrer b973c7ca27 RuboCop: Style/ParenthesesAroundCondition
[skip-stages=Flakey]

(auto-corrected)

this complements Lint/AssignmentInCondition

Change-Id: I8e621631cb8565cc6062398cfab714f61b485cad
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274747
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-30 12:59:18 +00:00
Jacob Burroughs 6ce8e2e14e Just upload raw assets to the CDN
Just let cloudfront compress the assets for us.  It can handle the brotli/gzip
switching natively now.

Change-Id: I3416d428403f0bfbec5a575ff1c095d2342a5e4c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247818
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
2021-09-29 16:49:10 +00:00
Cody Cutrer 58d528037e RuboCop: Lint/AssignmentInCondition
[skip-stages=Flakey]

(manual)

Change-Id: I855b13a7888f4926df42fe6306cc248e611efdf6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274712
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-29 03:38:39 +00:00
Cody Cutrer dbd8c46655 RuboCop: Lint
[skip-stages=Flakey]

auto corrections applied:
 * Lint/AmbiguousOperator
 * Lint/AmbiguousOperatorPrecedence
 * Lint/AmbiguousRegexpLiteral
 * Lint/DeprecatedClassMethods
 * Lint/DeprecatedOpenSSLConstant
 * Lint/NonDeterministicRequireOrder
 * Lint/ParenthesesAsGroupedExpression
 * Lint/RedundantRequireStatement
 * Lint/RedundantSafeNavigation
 * Lint/RedundantSplatExpansion
 * Lint/RedundantStringCoercion
 * Lint/RedundantWithIndex
 * Lint/SendWithMixinArgument
 * Lint/SymbolConversion

Change-Id: I222ec19978033544513bb99755994d109435abad
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274551
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-29 03:38:23 +00:00
Cody Cutrer 117d484c1c rubocop: disable Rails/ContentTag for migrations
Change-Id: I44d4c789fe539158a74143de6665caa0c6e73977
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274676
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-29 03:23:02 +00:00
Cody Cutrer 223aedadff rubocop: update to 1.21
[skip-stages=Flakey]

new cop autocorrected: Layout/LineEndStringConcatenationIndentation

Change-Id: Ib59a7fadeb2d9af68d90eb82b9ac855dead29121
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274524
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-28 02:40:46 +00:00
Cody Cutrer c65d57737a RuboCop: Layout lib
Change-Id: I0655d9a9d750f2debd6378b03d8ddc1403ebc31b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274158
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-22 20:01:52 +00:00
Cody Cutrer 3c41ab50d2 require redis/scripting for LocalRedisCache
it's not already required by inst-jobs anymore

Change-Id: Id2cfda3e0aa6ae0b2a3cdbf2931fb717e326cc5b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273419
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-09-13 14:29:22 +00:00
Jeremy Stanley 4ad0004d11 migrate pace plans
test plan:
 - create a pace plan in a course
 - set attributes on the pace plan
 - set durations for module items
 - copy the course
 - the destination course should preserve pace plan attributes
   and module item durations

flag = none
closes LS-2451

Change-Id: If13cbb61ddbd1edc5cc23fd90cd2f74e4a274179
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272073
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeff Largent <jeff.largent@instructure.com>
QA-Review: Jeff Largent <jeff.largent@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2021-09-02 22:19:43 +00:00
Angela Gomba a4f2b10532 Add OutcomeFriendlyDescription live events to Canvas
closes OUT-4695
flag=none

test plan:
- ensure live events are running locally (see live_events.md)
- verify that when making changes (including soft deleting) or
 creating an outcome friendly description that live events are emitted

Change-Id: I98ae5aeda1e16e1816f7d63a6a82f777a2766ae0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272071
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Augusto Callejas <acallejas@instructure.com>
2021-08-30 15:48:59 +00:00
Cody Cutrer 160b3c1f69 don't regenerate force-reuse access tokens for userinfo requests
i.e. instfs

in this case, we're never returning the actual access token, so as
long as the token isn't expired, it doesn't matter. and since force
reuse tokens _can't_ expire, it's fine

Change-Id: I8da6c21755586b25fddcf05df8e35dadf8610afa
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271998
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-24 18:27:11 +00:00
Cody Cutrer 31c7af578f remove SafeYAML
Psych has safe_load now, and it's fairly trivial to convert our existing
overrides to use that instead

Change-Id: I2648df8d4574e15fc9072a25882e318d902765c3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271939
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-24 16:58:54 +00:00
Cody Cutrer 25870990ac missed removing live events plugin registration
Change-Id: I7a0e5f11b1cd656178f8ab97306bd7383f7d2784
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271514
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-18 16:56:35 +00:00
Cody Cutrer b82c907ea0 don't use plugin settings for live events anymore
they've been in consul for a very long time, and it's far more
performant anyway

Change-Id: I81e58a0275c3cf502ce817cd6c66ed3080525a52
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271510
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-16 18:29:54 +00:00
Cody Cutrer 1ae9939a54 use request.host_with_port
not HTTP_HOST, since we need to take into account X-Forwarded-Host

also send the proxy headers to Sentry

Change-Id: Ic71bcdfd2a8f1d2ec6bee66c81b6d536f96d27c2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271329
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Nathan Mills <nathanm@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-13 03:18:01 +00:00
Cody Cutrer 68768f17b8 always regenerate the access token for force_token_reuse
fixes FOO-2265

no more returning no token, because we don't have it. it makes it impossible
to get it if you lost it. instead, if you're a force_token_reuse integration
you _must_ invalidate prior access tokens everytime you request a new one

Change-Id: I66d52279abf3d58e5a6d2c90d97007077a73e68d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271276
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-12 17:58:33 +00:00
Cody Cutrer 6c43507a4c fix reloader for Vault
Change-Id: I0ec72978cd1fc1cda8f7a84950e95fa95fa84912
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271104
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-10 22:37:42 +00:00
Cody Cutrer 8acc8f54a6 fix dynamic settings initializer
just load settings from consul.yml to avoid circular references at boot

Change-Id: I405e8541cc46e58f2447d33c87d99d6906da5ca9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271102
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-10 19:56:34 +00:00
Cody Cutrer b5945df961 handle consul retries down in DynamicSettings
so that everyone gets the benefits, not just instfs

also include a new circuit breaker so that if consul is unresponsive for more than the
retry interval, we just let failures through quickly for a while

Change-Id: I9ba757c8529c1011ca771612f592f289c6a844b6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/270789
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-10 15:19:33 +00:00
Cody Cutrer 1293cb152b lower the priority of CourseProgress.dispatch_live_event slightly
it's not technically user visible, so doesn't need highest priority.

Change-Id: Ib38e245bd00341930ef70b6702aac8d1a758fe6d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/270967
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-10 13:03:25 +00:00
Cody Cutrer 90b485d3d5 fix several Switchman::OrderOnMultiShardQuery
Change-Id: Iac1ff2683375c8f7f57f5ed929de47bebf71d129
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271003
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-10 12:06:35 +00:00
Cody Cutrer f5d0cc64bc add failsafe option for DynamicSettings
so that we're not re-implementing it at multiple callsites

also remove unused error classes

Change-Id: I938d705729f2208532b4522eddbc8edfa4f2031f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269561
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-08-02 18:27:59 +00:00
Xander Moffatt 6999c05304 debounce course_completed live events
* and change singleton key

why:
* catalog is getting much fewer course_completed live events than they
used to
* live events log diving reveals the events _are_ getting sent, but as
course_progress instead, even though the API shows that the course
has been completed
* the on_conflict option defaults to :use_earliest, which doesn't
completely explain these discrepancies, but certainly isn't helping,
since the docs for inst-jobs say to use on_conflict: :overwrite for
debouncing.
* also, uncovered that sometimes if multiple modules are completed in
the 2 min debounce period, multiple jobs will be enqueued since the key
for the singleton strand is the ContextModuleProgression id. It would
be better if the key used the course and user, so that only one job and
only one event get sent at a time.
* this isn't a final solution but is a helpful first step

closes INTEROP-6924
flag=none

test plan:
* in `live_events_callbacks.rb:159`, change the delay_if_production to
just delay, for testing
* follow the directions in `doc/live_events.md` to set up live events
piped to the canvas docker kinesis stream, and tail the stream to see
live events flowing (reminder: after configuring the Plugin you *must*
restart canvas).
* have a course with a student and 4 modules, each with one
assignment in them
* as the student, submit the assignment for the first module
* in a rails console,
`Delayed::Job.where(tag: "CourseProgress#dispatch_live_event")` should
show you the jobs enqueued for the student
* there should only be one, and the strand should include the global ids
for the course and user
* the run_at time should be for two minutes from the created_at time
* in the browser, submit the assignment for the second module
* the run_at time should update to two minutes from now
* wait two minutes and let the job run
* in the stream logs, you should see a course_progress event
* in the browser, submit the assignments for the last two modules
* another job should be enqueued, wait two minutes and let it run
* in the stream logs, you should see a course_completed event, and no
more course_progress events (besides the first)

Change-Id: Ib6860a185031a9b9f0375347e079a0501f3b571c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269276
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
2021-07-27 18:07:22 +00:00
Xander Moffatt 7096c0a96d allow site admin to get LTI token for any 1.3 tool
why:
* getting an LTI access token currently requires installing the LTI 1.3
test tool and configuring it then requesting a token,
or otherwise hacking other tools to make a request
to the token endpoint
* it can be very helpful for troubleshooting to have an
LTI access token for any needed tool
* note that 1.1 tools don't have DeveloperKeys, and 2.0 tools use their
own access tokens. this main use case is for 1.3 tools and for the AGS.

test plan:
* choose a 1.3 ContextExternalTool
* navigate to canvas.docker/api/lti/advantage_token?tool_id=<tool's id>
* it should return an access token in the form of a JWT
* choose a DeveloperKey that is an LTI 1.3 key
* navigate to canvas.docker/api/lti/advantage_token?client_id=<key's id>
* it should return an access token
* choose a DeveloperKey that isn't for an LTI tool
* this url should return 400 and say it must be a 1.3 key
* choose a ContextExternalTool that isn't 1.3
* this url should return 400 and say it must be a 1.3 tool
* sign out of Canvas
* this url should return unauthorized
* sign in as a non-site-admin user
* this url should return unauthorized

Change-Id: Ie599fe3c4a8413ad0a65515837360c439ddb3b9c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269109
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
2021-07-20 20:07:09 +00:00
Michael Ziwisky c44b1844f7 InstID tokens, part 1: generation
fixes INTEROP-6913, INTEROP-6892, INTEROP-6893, INTEROP-6920
flag = none

This commit introduces the InstID token, a signed and encrypted JWT (aka
JWE) that will soon be usable for Canvas API access (that's "part 2").

If the InstID class is configured with a private signing key and public
encryption key, it will be able to produce encrypted JWTs and validate
and deserialize decrypted JWTs.  If it is configured with only a public
signing key, it cannot produce tokens but it can still validate and
deserialize decrypted ones.  Therefore this class can be used by the
identity provider (currently Canvas) to produce tokens, but also by any
services that want to use InstID tokens for authentication.

test plan:
 1) generate two RSA keypairs.  one way to generate a keypair is from a
    rails console:
    > keypair = Canvas::Security::RSAKeyPair.new
    > puts keypair.private_key.to_s
    > puts keypair.public_key.to_s

 2) choose which one is for signing and which is for encryption, then add
    the private signing key and the public encryption key to your rails
    credentials:
    - run `bin/rails credentials:edit`
    - add an entry like the following, and then save and close your
      editor:

```
inst_id:
  encryption_key: |
    -----BEGIN PUBLIC KEY-----
    MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvY1EMlGm1daM87ejGuFX
    <...snip...>
    /wIDAQAB
    -----END PUBLIC KEY-----

  signing_key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEAnDwED/QOB0f0H6TOZqLmjaPqA7m8c40NDXkAa6u5cK8zCbk3
    <...snip...>
    QhjPgifBwTrzj21484CfiPfy5oe756Exerj8PIlRrE/hxWRSDwBIOg==
    -----END RSA PRIVATE KEY-----
```

 3) open a rails console and do:
    > id = InstID.for_user('user-uuid')
    > id.to_token # make sure this doesn't blow up
    > token = id.to_unencrypted_token
    > decoded_id = InstID.from_token(token)
    > id.jwt_payload == decoded_id.jwt_payload # => true

TODO in followup commits:
 - make canvas accept InstID tokens for auth

Change-Id: Ie550c17507c26f9944bd62a747a6a63161e8e770
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268872
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
2021-07-19 20:58:26 +00:00
August Thornton 2c77683ac0 remove granular permissions for course files FF
closes FOO-1766
flag = none

[fsc-max-nodes=18]
[fsc-timeout=30]

Test Plan:
  - Run the migration and make sure there are no errors
  - Some things to check:
    * How it acts as a teacher, student, and public user
      in course files/folders and personal files/folders
      with the various settings above toggled to different states
    * How it acts as a teacher, student, and public user
      in discussions, modules, content migrations/import/exports
      (RCE should behave similarly throughout the site)
    * Should only be able to upload or add folders if the
      Course Files - add permission is enabled for the user's role
    * Should only be able to manage file access, usage rights, move,
      or rename course files/folders if the Course Files -
      edit permission is enabled for the user's role
      • Check Toolbar header at the top of Course files
      • Check Cog (hamburger menu) to the right of each file/folder
      • Check Usage Rights Indicator under usage rights column
        that can be found in course and group file pages. This can
        be enabled under course settings if not available
    * Should only be able to delete course files/folders if the
      Course Files - delete permission is enabled for the user's role
    * Any given user/role should have full access to their respective
      personal files/folders regardless of granted permissions. The
      same also applies to a group context with some caveats
      • Should not be able to modify file access in a group context
      • Should not be able to modify usage rights in personal files
    * A student enrollment not granted any file permissions (the default)
      should only be able to _view_ and _download_ files unless granted
      additional access from an authorizing role
    * REST API works as expected
    * UI works as expected with no additional javascript errors
Change-Id: I5e7f717494d658e6c8ec9be8a8039015afcebc63
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262775
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
2021-07-16 15:31:37 +00:00
Jeremy Stanley 2244a69bcc parse wiki pages in migrations with HTML5
this matches the version used by Canvas proper when it
sanitizes user HTML, to prevent surprises in migrations

test plan:
 - put the HTML attached to the ticket in a page
 - it should survive an export/import or course copy round trip

flag = none
fixes LS-2241

Change-Id: Iae07c73e3b994dddf814ad5ee7dc4c2bd0284559
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267889
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2021-06-28 15:52:37 +00:00
Cody Cutrer 83730bad42 fix cache register reloading
code included/prepended into ActiveRecord::Base can't be reloaded, so
put it in a separate file that _won't_ be reloaded

Change-Id: I8f0e8993f101423284105b599bd6696a9c63eaec
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266169
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2021-06-03 17:03:05 +00:00
Jackson Howe dc5b874eb0 Tweak global nav strings for k5
Creates a helper in the application_controller called `k5_user?` which
returns true iff the user is associated with any k5 accounts. Also sets
a JS_ENV variable called :K5_USER with the value of ks_user?. Renames
other JS_ENV variables to :K5_HOMEROOM_COURSE and :K5_SUBJECT_COURSE for
clarity. Also removes references to 'k5_mode', replacing with calls to
k5_user? or calls on the Course model for clarity.

use_k5? is cached since it is frequently referenced. The cache key is
cleared for each user in the account when k5 mode is toggled by an admin
or when enrollments are adjusted. It is also set to expire after 1 hour,
which might not be necessary. Cache is not invalidated when the k5 flag
is toggled since the flag is being removed.

The user dashboard and global nav states are dependent on use_k5? Shows
`Subjects` instead of `Courses` and `Homeroom` instead of `Dashboard` in
the global nav when k5 is on. Also filters homeroom courses out of
course list when k5 is on.

closes LS-1827
flag=canvas_for_elementary

[fsc-max-nodes=20]
[fsc-timeout=40]

Test plan:
 - Login as a teacher enrolled in at least one course in a k5 account
 - Expect to see k5 dashboard
 - Expect to see `Subjects` and `Homeroom` (with home icon) in the
   global nav (and in the mobile global nav)
 - Toggle the courses/subjects tray in global nav and expect to see
   only references to Subjects
 - Open /courses/ and expect the page heading to be `Subjects`
 - Browse through the course and expect it to look normal (w/ Balsamiq)
 - Switch to a student and expect the above to be true, plus the courses
   list should not include homeroom courses
 - Disable k5 for the account and return to teacher or student
 - Expect to immediately see classic canvas nav, dashboard, and courses
   again
 - Test caching by placing a log statement (i.e.
   `p "K5 Cache: uncached"`) in ApplicationController#uncached_use_k5?
 - Run `redis-cli FLUSHDB`
 - Navigate around canvas; expect to see the log statement only once in
   the logs until toggling k5 mode or adding the first k5 course to a
   user

Change-Id: Iec970c939b976d10eee13d40b7bf68f601d8178e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265239
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeff Largent <jeff.largent@instructure.com>
QA-Review: Jeff Largent <jeff.largent@instructure.com>
Product-Review: Peyton Craighill <pcraighill@instructure.com>
2021-05-26 15:37:55 +00:00
Jacob Burroughs bca8bd4629 Migrate to diplomat
Change-Id: I82363dedbd2397830a9beadf33b1cc59123136c8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263429
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
2021-04-29 17:00:51 +00:00
Ethan Vizitei d815898134 make sure pulsar cert is on disk
and that we have a vault token

refs FOO-1842
flag=none

If a cert url is in the config, we need
to fetch and write it to disk before trying
to open a connection to pulsar or we'll
get failures.

Also add some config caching so
we aren't re-parsing the same YAML
blob over and over.

TEST PLAN:
  1) specs should pass
  2) be in a pulsar-enabled environment
      with a cert configured
  3) when you build a message_bus producer,
     the cert is fetched and written to
     disk at the configured path.

Change-Id: I2fb61badd05aedadb6d07475875692b30dd30db9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263687
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2021-04-28 15:36:05 +00:00
Jacob Burroughs 088cbb4cd5 Introduce release notes data model
Single dynamo table for all release notes data.  Should support active-record
style patterns (mostly) for CRUD operations, plus querying the latest N visible
records by role and environment.  Also supports paginated listing of *all* notes
for administrative purposes

fixes FOO-1752

Change-Id: Ic1e7e204e93e263479a738af18daf312c801269c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261548
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2021-04-05 16:09:59 +00:00
Han Yan 1af6e05426 add a field to content_migration_completed live events
refs QUIZ-8160

flag=none

test plan:
- test with quiz_lti patch /quiz_lti/+/261845

Change-Id: Ib8e1f8f2f5449b8ebbadea6067f8aaeef925c8f9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261846
Reviewed-by: James Logan <james.logan@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
Product-Review: Susan Sorensen <susan.sorensen@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2021-03-31 19:31:13 +00:00
Cody Cutrer e783fa5d2a add frozen_string_literal comment to straggling lib files
mostly rake tasks

Change-Id: Ia354de53f66b0707edac6db56a296c9005351922
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261817
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2021-03-30 18:13:54 +00:00