When updating a course's availability settings, we touch all of the
users in the course. This causes locks in production when there's
lots of users enrolled in several large courses being updated. In
the case where we are updating availability settings, we'll skip
touching user objects if the row is locked - another process will
update the timestamp.
flag = none
closes LS-3018
Test plan:
- In a rails console, get a relation of courses doing something
like `relation = Course.where(id: [1, 2, 3])`
- Run `relation.touch_all_skip_locked`
- Expect the updated_at timestamp of each course to be updated
- Check out the SQL that was executed and expect to see
'FOR NO KEY UPDATE SKIP LOCKED'
- Now run `relation.touch_all`
- Expect the timestamps to be updated again, but the SQL should
not include 'SKIP LOCKED' lock type
Change-Id: If499982dc8fa368d312da7b8bcfc61a1f28b773b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/286195
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jackson Howe <jackson.howe@instructure.com>
Product-Review: Jackson Howe <jackson.howe@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Makes DueDateCacher and EffectiveDueDates handle being passed global
ids. Note that these IDs must still be for records on the same shard
as the provided course.
closes EVAL-2207
flag=none
Test Plan:
1. Create a course in an account in a cross-shard trust
2. Enroll at least one teacher and one student
3. Create an assignment in the course with a due date
4. As the teacher:
i. Log into the trusted account (not the one with the course)
ii. Go to the calendar and move the assignment to a different day
(drag and drop, or edit)
5. Check via api that the submission is not deleted
Change-Id: I3b450a959e660e2aa50428b419bfa0298e7c471b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/285004
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
Reviewed-by: Dustin Cowles <dustin.cowles@instructure.com>
Reviewed-by: Syed Hussain <shussain@instructure.com>
Change-Id: I89129633731a68c38a5026b6b26318d1f3699a2a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284968
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Migration-Review: Jacob Burroughs <jburroughs@instructure.com>
There was already a Setting to control the frontend error sample rate,
so this change adds one to control the backend sample rate, as well, and
removes the unnecessary `sentry_disabled` Setting.
flag=none
closes DE-1034
test plan:
- verify that the backend error sample rate defaults to `1.0` when unset
- verify that setting it to `0.0` results in no backend errors being
collected
- verify that changing the setting takes effect without server restart
when a SIGHUP is sent (test by calling `Canvas::Reloader.reload!`)
Change-Id: Ib8905ef4d18da7e1d2647ed9122218f56499670e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284796
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Isaac Moore <isaac.moore@instructure.com>
Product-Review: Isaac Moore <isaac.moore@instructure.com>
Otherwise, methods like `Sentry.set_tags` will be delegated to
`NilClass`, and will fail when called. We wrap the initialization in a
rescue block so we don't break the app if this default init fails.
flag=none
closes DE-1016
test plan:
- remove `sentry.yml`
- verify app starts correctly, calls to `Sentry.set_tags` don't raise
- verify that with `sentry.yml` present and `sentry_disabled` setting
false, Sentry captures errors
Change-Id: I93c6e65f8aa08d1e52c0a05b8ffd23fc7f3ef4fa
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283868
QA-Review: Isaac Moore <isaac.moore@instructure.com>
Product-Review: Isaac Moore <isaac.moore@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
new tables can be created in a transaction, but existing tables with
many rows need to have indexes added concurrently outside of a
transaction. adding a replica indentity requires the addition of an
index. let's peek the table and see if there are any rows to determine
how we need to have this index added.
Change-Id: Ieff6d8ad4d8d95b27b281ce2b96cacb2f390df48
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283508
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ben Rinaca <brinaca@instructure.com>
Product-Review: Ben Rinaca <brinaca@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
i wanted to make a change, and there weren't specs here to begin with.
Change-Id: I048e2d512df940607e47ea4add846da7ed876002
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283507
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Ben Rinaca <brinaca@instructure.com>
Product-Review: Ben Rinaca <brinaca@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
* respect truthiness of AR_QUERY_TRACE env var (setting it to
"0" or "false" turns it off)
* allow AR_QUERY_TRACE in test as well as development
* add AR_QUERY_TRACE_LEVEL option (valid values are "app", "rails",
and "full")
Change-Id: I398b800bb42b1bc7716cf322a6e6330bfaa8955b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278849
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
[skip-stages=Flakey]
auto-corrected, with post-review changing hashes that were meant
to be string keys in the first place
Change-Id: I877a365b9035bb62cea4d3b2f01f641f55b63281
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278676
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
flag=none
Test Plan:
- Set the "AR_QUERY_TRACE" env var to "true"
- Restart Canvas
- Load a Canvas page and verify query traces
are being logged
- Set the "AR_QUERY_TRACE_LINES" env var to some number
- Restart Canvas
- Load a Canvas page and verify the query traces
are limited to the number you chose
- Set the "AR_QUERY_TRACE_TYPE" env var to either
"write" or "read"
- Verify only queries of the chosen type are logged
- Verify query traces are not logged in in non-
development environments ("test", for example)
Change-Id: I2f5f7610a71e0bbfaa80327c422fc87162a5a78b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278634
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
[skip-stages=Flakey]
all manual
the fixes are a little scattered, since the same method doesn't work
everywhere depending on requirements. mostly I changed to `let`, but
some required `stub_const`. For `let`, I eventually settled on
avoiding a dedicated `let` for the class if it's only used one, and
it's a trivial class just to include the module. otherwise there's
a separate `let` for the class, and if there's only one it's named
`klass` instead of something contrived.
Change-Id: I84734c963d4789be3ec3cd852cca623e7c2a08df
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277285
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
and prevent it in test env (it's still useful in console and scripts for
dev and prod)
Change-Id: Iaf77dc91c5fff226eebaf23a3b3cb9c2c4af8b94
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275236
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-2410
test plan:
- in dynamic_settings.yml, add the following block:
```
store:
canvas:
services-jwt:
# these are all the same JWK but with different kid
# to generate a new key, run the following in a Canvas console:
#
# key = OpenSSL::PKey::RSA.generate(2048)
# key.public_key.to_jwk(kid: Time.now.utc.iso8601).to_json
jwk-past.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-05-18T22:33:20Z_a\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
jwk-present.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-06-18T22:33:20Z_b\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
jwk-future.json: "{\"kty\":\"RSA\",\"e\":\"AQAB\",\"n\":\"uX1MpfEMQCBUMcj0sBYI-iFaG5Nodp3C6OlN8uY60fa5zSBd83-iIL3n_qzZ8VCluuTLfB7rrV_tiX727XIEqQ\",\"kid\":\"2018-07-18T22:33:20Z_c\",\"d\":\"pYwR64x-LYFtA13iHIIeEvfPTws50ZutyGfpHN-kIZz3k-xVpun2Hgu0hVKZMxcZJ9DkG8UZPqD-zTDbCmCyLQ\",\"p\":\"6OQ2bi_oY5fE9KfQOcxkmNhxDnIKObKb6TVYqOOz2JM\",\"q\":\"y-UBef95njOrqMAxJH1QPds3ltYWr8QgGgccmcATH1M\",\"dp\":\"Ol_xkL7rZgNFt_lURRiJYpJmDDPjgkDVuafIeFTS4Ic\",\"dq\":\"RtzDY5wXr5TzrwWEztLCpYzfyAuF_PZj1cfs976apsM\",\"qi\":\"XA5wnwIrwe5MwXpaBijZsGhKJoypZProt47aVCtWtPE\"}"
```
- Ensure /internal/services/jwks loads correctly
- In console, ensure `CanvasSecurity::ServicesJwt.decrypt(Base64.decode64(CanvasSecurity::ServicesJwt.for_user('localhost', User.first)))`
and `CanvasSecurity::ServicesJwt.decrypt(Base64.decode64(CanvasSecurity::ServicesJwt.for_user('localhost', User.first, symmetric: true)))`
both work and produce sensible looking output
Change-Id: I13c6c35cc92ed12d03bf97e89e590614e11c6d47
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275160
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
* clean up asset string handling in Context
* clean up the main cache key
* hash up the long options hash
* use an integer for the timestamp, rather than letting it convert
to a bunch of segments
* reduce Context.last_updated_at to a single query by using UNION
* if a user is given, just start with the list of contexts we would show,
then reduce by what they're asking for. this is simpler and far more
performant than doing a ton of permissions checks
Change-Id: I00223dc162f51f5541dc3468a839333fd6bf32a6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271665
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
* don't get rid of the select in a FROM union, it might be on purpose
* limit and order can be used, but if you do you need to enclose each
subquery in parentheses; just do it all the time
Change-Id: I33311d702dddc1d82fb5bee2eaea93f78e50e2fe
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271663
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs INTEROP-6890
TEST PLAN:
1) insert base64 encoded keypairs to the
inst_access_signing config file
2) start your server
3) it does not explode
Change-Id: I286bf1e335d7ceb95df4f847f9e96231ba1336bf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/270905
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
This reverts commit 9f7868a1e5.
Reason for revert: copy strategy should be fixed
Change-Id: I303966bb2a48f12496064329fc03179915411615
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268391
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
This reverts commit 2c5c3584ff.
Reason for revert: I'm back in the office and can debug the problems
Change-Id: Ib469fff450a8d51d7ca59cb9d7fa29874d6b6e53
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268386
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
This reverts commit 230033611d.
Reason for revert: COPY strategy is broken
Change-Id: I2cb4e5430c180caa1ceae6570d27a4c86a12d704
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267654
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
This reverts commit d3ced99046.
Reason for revert: COPY strategy is broken
Change-Id: Iba262068e93ddd10e1a15c123c37f5cd97e91c2d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267656
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
* do everything as in_batches, returning a relation. properly
super `load` param in each backend
* plumb strategy through all entry points so it can be explicit
* special case in_batches.delete_all with no explicit strategy to
do a loop on a limited delete_all (avoids a dumb ORDER BY, or
having to transfer ids back and forth)
* since in_batches can easily be used with pluck now that a relation
is returned, just make find_in_batches_with_temp_table a shim that
does it the "nice" way
Change-Id: I716f188cdf676a725588f94a1036981ae798b09c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266882
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Change-Id: I583b273fb474960d37a30e605ff232d1a5c93b80
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262409
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
closes INTEROP-6661
flag=none
* adds a new active record helper, with_statement_timeout
test plan:
* to test the timeout helper in a rails console
```
ActiveRecord::Base.with_statement_timeout(1_000) do
ActiveRecord::Base.connection.execute("SELECT pg_sleep(3)")
end
```
* ^ that should error with ActiveRecord::QueryTimeout
* in a course with content migrations imported, launch an LTI tool
that requests the `Canvas.course.previousContextIds.recursive`
variable substitution
* that variable should not change whether this commit is checked out
or not
Change-Id: I65a9ebf9644aa8a33986a4a2089362af72e74cac
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261772
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Per-database-server offsets will be added in a separate commit
refs FOO-1600
Change-Id: I836691ba18b373717f139e116b9b91376f67b682
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260356
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Jackson Howe