Commit Graph

1752 Commits

Author SHA1 Message Date
Ethan Vizitei 2cf7f03656 cleanup AUA log compaction
and fix bug with zeroing out
wrong iterator

Change-Id: I7e94d3d256c128b690c9b6fb2df038dca449ee3a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247090
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2020-09-06 13:10:35 +00:00
Ethan Vizitei 08e2238dc6 aua views via log compaction
closes FOO-937

 * 7-tables by day-of-week
 * infer write table from timestamp
 * plugin with default settings values
 * instructions on setting the local shard to use log-&-compact
 * write path switches on setting to log compaction for view bumps
 * compaction job
 * single write per log segment (multpile updates)
 * truncates previous partition
 * periodic job harness with available settings

Change-Id: Idd5cbaf0d83436724c5d72038e0dbc78cdc23619
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/247083
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2020-09-06 03:56:53 +00:00
Evan Battaglia 575c8cdbb0 Have backfill migration run for new tables
refs: INTEROP-6153
flag=none

Test plan
- N/A

Change-Id: I8c773073969975dc0f69725bc9437ba8e01b40d0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246952
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
2020-09-03 22:29:11 +00:00
Charley Kline 4cb9e145ab Migration: remove view_learning_analytics overrides
Refs FOO-905
flag=none

We've removed this permission from Canvas, so let's just also
go remove all role overrides everywhere that reference it.

Test plan:
* If you had any RoleOverride records referencing the
  :view_learning_analytics permission, they should be gone after
  running DataFixup:ClearAnyCanvasInsightsViewRoleOverrides.run

Change-Id: I028eaa929dc4ae1176b683980ad2d3cb07c2dfbf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246805
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-09-03 19:31:38 +00:00
Ethan Vizitei ea14dcca1b optimize most time-consuming production query
Change-Id: Ia0e8eff177df99cd24d55c3e113d84925a081723
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246914
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2020-09-03 17:42:49 +00:00
James Williams a6e6fd0670 split built-in role rows by root account
test plan:
* a multi-tenant shard should still work
 creating enrollments and account admins before and after
 running postdeploy migrations
* the postdeploy should successfully run a datafixup
 to split up build-in role rows and reassign the role_id on
 other tables (account_notification_roles, account_users,
 enrollments, and role_overrides) accordingly

closes #INTEROP-6045

Change-Id: I8325acb635397ac1b008b9d09788595da597cec7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243906
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
2020-09-01 19:21:27 +00:00
Ethan Vizitei 16abf15de4 yank migrations and table management from canvas ddb
Change-Id: I3d2a60443e9b514f690a236b05dd2f437ad8e3c2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246391
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2020-08-31 20:26:17 +00:00
Cody Cutrer bedf970aab optimize planner wiki page query
it's far more common for wiki pages to not be linked to wiki pages, so only look up their
assignment visibilities if necessary

Change-Id: Idf3f05e2c5e59a2163bd692497160c2388551be2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246533
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2020-08-31 16:53:05 +00:00
Cody Cutrer 2af7da3928 add index on account_report_rows(created_at)
so that clean up is much faster

Change-Id: I265091dc328a54fb849b3eff39cad685e7889ae1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246525
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2020-08-31 16:22:55 +00:00
Mysti Lilla 66115e206c Revert "Revert "Create dummy Account 0""
This reverts commit 2548a2bc38.

Reason for revert: <INSERT REASONING HERE>

Change-Id: Icfb9e1a2bcc660b66b8ee740299d390ca844c261
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245091
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
2020-08-28 20:57:49 +00:00
Mysti Lilla 2548a2bc38 Revert "Create dummy Account 0"
This reverts commit 2a7fcdd792.

Reason for revert: <INSERT REASONING HERE>

Change-Id: I0f75b86493f5b24b91cee2b00ec9e0c07d496d49
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245090
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
2020-08-28 18:22:13 +00:00
Mysti Lilla 2a7fcdd792 Create dummy Account 0
flag=none
closes INTEROP-6094

Test plan
- Run the database migration
  and ensure Account 0 is created

Change-Id: I9a5e6f8934c76ea97997a41eeecdd7fef7e76e95
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245895
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Guymon <mguymon@instructure.com>
Product-Review: Michael Guymon <mguymon@instructure.com>
2020-08-28 16:17:09 +00:00
Jacob Burroughs ae5ab6bfb6 Bump inst-jobs + switchman-inst-jobs
Change-Id: I69c7a60bf65f1ef8914171413adb91bb2eae95a5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/246089
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2020-08-27 16:25:49 +00:00
Adrian Packel abe50c3b3c Add grading_period_id to Cassandra
flag=none
closes EVAL-1128

Test plan:
- Migrations on Cassandra run successfully
- Grade changes save the grading_period_id of the associated submission
  in both Cassandra and AR

Change-Id: I7ae0ff54093b9bf1eaea43106d4788f22c6c920e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245734
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Adrian Packel <apackel@instructure.com>
Product-Review: Jody Sailor
2020-08-26 19:21:32 +00:00
Mysti Lilla 3251306ff0 Delete DeveloperKey shard split artifacts
refs INTEROP-6080
flag=none

Test plan
- Have some developer keys with account ids
  from a different shard
- Run DataFixup::CleanupCrossShardDeveloperKeys.run
  and verify they get deleted

Change-Id: Ib389ef37b33a8a6e73c4926d9db775542dc6629f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245434
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
2020-08-20 20:01:21 +00:00
James Williams 4403eb3c2a fix cross-shard content shares
shares belong to the user's shard, not the content export's

so shares created for a course on a shard
that the user doesn't belong to will break

and shares sent to cross-shard users that create "received"
copies of the original share will also break

also semi-inadvertently fixing share sender info in API
and some problems on the import side when the correct
shard isn't active

test plan:
* cross-shard content shares should work
 (both when sharing content from a course from a shard
 the sending user is not native to, and when sharing content
 to a user on a shard different than the content's course)

closes #LS-1337

Change-Id: Ib91529ffa5de0ee7a6f20f63281d6c5508982063
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245496
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
2020-08-20 17:07:08 +00:00
Cody Cutrer ecd05261cc filter assessor asset by courses too in submissions_needing_peer_review
test plan:
 * specs pass

Change-Id: Ib66137d96d8ae06634f4e305b9354b306a0a2405
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245475
Reviewed-by: James Williams <jamesw@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-08-19 16:28:49 +00:00
Cody Cutrer 45aa6d632f tweak indexes for dashboard queries
Change-Id: I8e35155c8e1cff3be11db8c0de9fe9502951c6e1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/245106
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2020-08-14 18:52:04 +00:00
Pat Renner 466271355c soft delete outcome proficiency and proficiency ratings
closes OUT-3841
flag=none

test-plan:
- run migrations
- Ensure you can soft delete outcome proficiencies
(and their associated ratings)
- dc run --rm web script/rails c
> o = OutcomeProficiency.first (alternatively,
copy the factory code from factories/outcome_proficiency_factory
to create one for an account)
- ensure it has a workflow state (should be "active" by default)
- ensure the ratings have an "active" workflow state by default
> o.outcome_proficiency_ratings.map(&:workflow_state)
> o.destroy
- Ensure the outcome proficiency was soft deleted
(it should now have a workflow state of deleted)
- Ensure the ratings were soft deleted
> o.outcome_proficiency_ratings.map(&:workflow_state)
- Ensure calling undestroy restores both the proficiency and the ratings
> o.undestroy
> o.outcome_proficiency_ratings.map(&:workflow_state)
- Calling destroy_permanently! should hard delete
both the proficiency and all associated ratings (examine SQL output)
> o.destroy_permanently!

Change-Id: I6421fdfdd97b991e76fb08b528cddbc668495ac8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244166
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Augusto Callejas <acallejas@instructure.com>
2020-08-13 23:50:03 +00:00
Cody Cutrer 8ce6b71a3a optimize assignments_needing_grading_query
especially for courses with mostly-deleted-assignments

Change-Id: Icbb010dd0406e2206148ce08ee849babf4d7a459
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244991
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2020-08-13 20:34:53 +00:00
Jacob Fugal 2b3886c5f4 extend client_credentials oauth2 grants for CD2
refs SAS-1540

* adds an audience setting to developer keys, so a key can be set to
  target external audiences with its credentials grants
* when a key with an external audience grants credentials, the token is
  signed with an asymmetric key instead of the internal symmetric key
* external audiences can retrieve the corresponding public keys from
  /login/oauth2/jwks
* credentials issued by developer keys with an account id include the
  account's guid in a custom claim

includes a refactor of key storage and rotation in consul, which had
already been done for LTI. but it wasn't really a feature of lti, just
something used by LTI, and we needed the same for key management for
this. moved it to be part of Canvas::Security

Change-Id: Ie5c0fcee6fc21687f31c109389a3bcc1ed349c5d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243606
QA-Review: Jonathan Featherstone <jfeatherstone@instructure.com>
Reviewed-by: Jonathan Featherstone <jfeatherstone@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
2020-08-13 18:52:47 +00:00
Evan Battaglia efe298c1c8 Clear old (always global) user root account ids
Must be merged and deployed with the "Make Users root_account_ids
relative to user shard" (g/244742) commit.
The migration here is named to run before
20200806175207_populate_root_account_id_on_models
kicks off the users backfill.

flag=none
refs INTEROP-6063

Test plan:
- make sure you have some users with root account ids with all global
  ids (ids are still global when on same shard as user)
- run migrations
- make sure those root account ids are cleared (if you are running jobs
  you will also start backfills so you may notice users filling up again
  but they will have local root account IDs for root accounts on the same
  shard as the user)

Change-Id: I047afe45989f2e86ca711e24683beca4557ce1c6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244804
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
2020-08-12 15:22:31 +00:00
Mysti Lilla 6b10d90197 Populate root account id on roles
fixes INTEROP-5887
flag=none

Test plan
- Ensure you have a role without a root
  account id
- Run DataFixup::PopulateRootAccountIdOnModels.run
- Ensure your roles have their root account ids
  populated

Change-Id: I9fd7fa4ac40b92716b746f0e7424b85212f118ff
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244525
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
2020-08-07 21:05:15 +00:00
Pat Renner 5949862ab8 ensure outcome_proficiency.context cannot be null
closes OUT-3805
flag=none

test-plan:
- run migrations
- cannot create an outcome_proficiency with nil context

Change-Id: Ie01742f0df539e7f6b8f31e1fedd80cd31eadfed
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242529
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Pat Renner <prenner@instructure.com>
2020-08-06 15:11:36 +00:00
Pat Renner 40fa8bd397 ensure calculation_method.root_account cannot be null
closes OUT-3847
flag=none

test-plan:
- Run migrations
- Ensure you cannot create a calculation method without a root account:
> OutcomeCalculationMethod.create! context: Account.find(id),
      calculation_method: :latest, calculation_int: nil
> method = _
- Confirm root account is set on method
- Update root account to be null (calling update_column skips
activerecord callbacks)
> method.update_column(:root_account_id, nil)
- Should trigger NotNullViolation

Change-Id: I1fa31c39f79d969e915872a2b420a6b583564be8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243759
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Pat Renner <prenner@instructure.com>
2020-08-06 15:11:25 +00:00
Pat Renner e846eaec57 remove account_id from outcome_proficiency model
closes OUT-3760
flag=none

test-plan:
- Run migrations
- Ensure outcome proficiencies cannot be made
without defining context (or context_type/id)
> rating1 = OutcomeProficiencyRating.new(
description: 'best', points: 10, mastery: true, color: '00ff00')
> rating2 = OutcomeProficiencyRating.new(
description: 'worst', points: 0, mastery: false, color: 'ff0000')
> OutcomeProficiency.create!(
outcome_proficiency_ratings: [rating1, rating2])
- Should pass with provided context
(may need to delete current proficiency ratings for Account.first)
> OutcomeProficiency.create!(
outcome_proficiency_ratings: [rating1, rating2], context: Account.first)

Change-Id: I67ea2237641e02e3339dbb7c468f9e91381422b2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242532
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Brian Watson <bwatson@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Pat Renner <prenner@instructure.com>
2020-08-06 15:11:15 +00:00
Evan Battaglia a45b1e1036 Add index to users root_account_ids
Needed for communication_channels backfill (it depends on users being
backfilled to needs the index to efficiently check).

refs INTEROP-6038
flag=none

Test plan:
- run migration up, check index exists on table
- run migration down, check index no longer on table

Change-Id: I27a5001a42bbd52cf630204148cd64900c80a1d5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244273
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
2020-08-05 21:54:18 +00:00
Mysti Lilla 05b02fc11c Have backfill migration run for new tables
refs: INTEROP-5842
flag=none

Test plan
- N/A

Change-Id: If7459b1890c25c00194a429b528a5b1aeeef8eaa
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/244282
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
2020-08-05 17:28:12 +00:00
Adrian Packel 72620b417b Update grade change tables for override grades
flag=none
closes EVAL-1065

Test plan:
- "up" migration:
  - Ensure migrations run properly
  - After the migration finishes, check that the assignment_id,
    submission_id and submission_version_number columns in the
    auditor_grade_change_records_* tables do *not* have not-null
    constraints
  - Check that the grading_period_id column exists on said tables, along
    with an accompanying foreign key
- "down" migration:
  - Check that the assignment_id, submission_id and
    submission_version_number column once again have a not-null
    constraint
  - Check that the grading_period_id column no longer exists

Change-Id: I82d0325ea88f88138e78901415ed8fbe0d7d2814
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243691
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
QA-Review: Adrian Packel <apackel@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
2020-08-04 14:41:30 +00:00
Evan Battaglia 05e2c90f6b Remove RA id FK constraint on asset_user_accesses
Although we're not sure why, there are lots of asset_user_accesses with
a cross-shard context and thus would have a cross-shard root account ID.

flag=none
closes INTEROP-6042

Test plan:
- run migration and make sure you can set root_account_id to a global ID e.g.
  12340000000000123 on a AssetUserAccess
- set it back to null
- run `rake db:migrate:down VERSION=20200730144201` and make sure that you
  cannot set root_account to a global ID anymove

Change-Id: Idf1db168c52e0d76bdaeed9e02466fe465276e89
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243922
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
2020-08-03 21:19:06 +00:00
wdransfield 1a78816bab Bump date of root_account_id backfill migration
Closes PLAT-6031
flag=none

We have several new root_account_id population
data fixups. This date change will ensure the
new fixups included in the same release are
scheduled to run.

Change-Id: I45490e6a2dacb182de48df54b33ced658aa7e436
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243485
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
2020-07-27 19:30:18 +00:00
Michael Brewer-Davis cd9eb4fab7 add outcome calculation method model
closes OUT-3759
flag=account_level_mastery_scales

Test plan:
- run migrations
- in the canvas ui or rails console, create subaccounts
- in the rails console, add outcome calculation
  methods to the root account and some of the subaccounts:
  > ap Account.all.pluck(:id, :name)
  > OutcomeCalculationMethod.create! context: Account.find(id),
      calculation_method: :latest, calculation_int: nil
- verify that calculation methods are found through
  `account.resolved_outcome_calculation_method`

Change-Id: Iba4b8db2e1c78b7f7c5964eec270ed75c4f94d66
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242745
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Augusto Callejas <acallejas@instructure.com>
2020-07-22 17:53:42 +00:00
Evan Battaglia 005df77737 Remove root_account_id from communication_channels
This was replaced with plural root_account_ids.

Test plan:
- run migrations and check that communication_channels no longer has
  root_account_id column
- `rake db:migrate:down VERSION=20200713124612` and check that
  root_account_id column is there again

Change-Id: I6524ed6230687c8866e714a57c366bc31aa1d5a6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242524
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
2020-07-22 15:55:11 +00:00
Pat Renner 49b9636f7f add context_type/id to outcome proficiency
flag=none
refs OUT-3758

DO NOT MERGE (migration freeze)

test-plan:
- ensure you have at least one outcome proficiency
created for an account
- run migrations
- ensure that for your existing outcome proficiencies,
context_type and context_id are set
- create an outcome proficiency (using
outcome_proficiency_factory provides a nice template) and
ensure context_id, context_type is set on save

Change-Id: Iabd21ce00386959f12498f7fa0c1f5bd58d57127
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242140
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
2020-07-22 15:09:08 +00:00
Drake Harper 079860ea4d Drop gender and birthdate columns from User
Test Plan:
-specs should pass

flag=none

fixes VICE-591

Change-Id: I20595e1f7aefe8e4c6237db60b3eac9139780fef
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242087
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2020-07-20 17:49:46 +00:00
Xander Moffatt 24153c85c9 backfill conversations_* tables
* read from `conversations`, which already has root_account_ids
* copy those, which are comma-separated values in a string, to:
  * conversation_messages
  * conversation_participants
  * conversation_message_participants, through conversation_messages

closes INTEROP-5853, INTEROP-5854, INTEROP-5855, INTEROP-5856
flag=none

test plan
* specs
* in the UI, create a conversation chain between two users
* in a rails console, note that the conversation has a root_account_ids field
* note that the Conversation's `conversation_messages`,
`conversation_participants`, and
`conversation_message_participants`
all have those root account ids as well, since there is logic to populate those on save
* for each of those associations, run `association.update_all("root_account_ids=NULL")`
* then run `DataFixup:PopulateRootAccountIdsOnConversationsTables.run`
* note that the root_account_ids fields for all associations
match the field on the conversation

Change-Id: I98ac281291c2105240610b0a0690a544c7d082a7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241978
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
2020-07-16 17:24:56 +00:00
Cody Cutrer 74d6822704 disable pg12 COLLATION for the moment
it doesn't work well in a mixed environment

Change-Id: I90b72e7a0fce1a9e4f815cc89f75b9a6b258ddfa
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242679
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2020-07-15 00:19:33 +00:00
Xander Moffatt 81f3dbad4f populate root_account_id on calendar_events
closes INTEROP-5844
flag=none

test plan:
- have existing CalendarEvents without root account id,
  preferably at least one with each context type
  (Course, CourseSection, Group, User w/no effective
  context, User w/effective context, AppointmentGroup)
- run `DataFixup::PopulateRootAccountIdOnCalendarEvents.run`
- verify all events (except User w/o effective context)
  have a root_account_id

Change-Id: I7d3c3c7221c25b3ae2d8c1b8819307cc66be229e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242024
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
2020-07-14 19:28:38 +00:00
James Williams d3189e9f01 fix remove_foreign_key with if_exists and column
refs #LS-1201

Change-Id: I4240f4b642b1c99bb6dfe3ce88713b597797bea1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242590
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Keith Garner <kgarner@instructure.com>
Product-Review: Keith Garner <kgarner@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
2020-07-14 14:56:47 +00:00
Evan Battaglia 0c876829d4 Remove FK constraints for more global RA ids
root_account_id on these tables can be cross-shard, so cannot have an FK
constraint.

flag=none
closes INTEROP-6006

Test plan:
- run migration and make sure you can set root_account_id to a global ID e.g.
  10000000000001 on a AccessToken and RubricAssociation
- run `rake db:migrate:down VERSION=20200713144312` and make sure that you
  cannot set root_account to a global ID anymove

Change-Id: I5ea00506f9eacba4ff0106133bcd0e8dcbaa6dc1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242539
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-07-13 22:13:24 +00:00
Evan Battaglia 386c44e691 Add root_account_ids to communication_channels
communication_channels can have multiple root account IDs
(they can have no pseudonym and instead just belong to a user,
which can have multiple root account IDs).

flag=none
refs INTEROP-6003

Test Plan:
- Verify the new migration can be run up and down.
- Verify the root_account_ids attribute on the CommunicationChannel
  model can be set and saved.

Change-Id: I8009ff62c3560d1de5d761ddf8d8a042b48327ca
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242522
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2020-07-13 21:06:54 +00:00
Mysti Lilla 597f0a7274 Ensure backfill job runs per shard
fixes INTEROP-5997
flag=none

Test plan
- Specs pass

Change-Id: I857548924a618eeb068e129e16292ced6494abba
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242414
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
2020-07-13 20:15:16 +00:00
Evan Battaglia cc23faaac3 Remove FK constraints for possibly global RA ids
root_account_id on these tables can be cross-shard, so cannot have an FK
constraint.

Test plan:
- run migration and make sure you can set root_account_id to a global ID e.g.
  10000000000001 on an Attachment, AttachmentAssociation, Folder,
  CommunicationChannel (this one you need to use update_all() with SQL
  to avoid validations apparently)
- run `rake db:migrate:down VERSION=20200713114812` and make sure that you
  can not set root_Account to a global ID anymove

Change-Id: Ibcb8b54d1ba3af96bf4fad64a8ea3b7e4a613c2e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242500
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-07-13 20:00:21 +00:00
Clint Furse 54212b43da data fixup for MC LTI tool
closes LS-1182
flag = submission_type_tool_placement

Test Plan:
- spec runs ok

Change-Id: I36eb9050e9dc4114cad87ebf4101413d69d7b93e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242136
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Clint Furse <cfurse@instructure.com>
2020-07-13 16:16:08 +00:00
Gary Mei 42cb2c1529 add root_account_id to grading_period_groups
closes EVAL-702
flag=none

Test Plan
- Run migrations.
- Create a grading period group for an account.
- Verify in a console that the group's root_account_id is set to the
  root account's id.
- In the console, set the root_account_id to nil.
- Edit the grading period group in some way and save it.
- Verify in the console that the root_account_id is set again.

Change-Id: Ic38d227066297d5cc247aab37760f9c6df958d2b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/239943
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Syed Hussain <shussain@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Spencer Olson <solson@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
2020-07-13 15:29:45 +00:00
James Williams d99b163828 remove root account foreign key for favorites
closes #LS-1201

Change-Id: If55f2237528e855e2cb41314a412c0935fc117f4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242375
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
2020-07-10 17:52:47 +00:00
Mysti Lilla 54a51a11f9 Performance improvements for backfill jobs
refs INTEROP-5836
flag=none

Test plan
- Specs pass

Change-Id: I41adaffe60c359537d8baaad4a4d3d60daaa9333
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242322
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-07-10 01:08:43 +00:00
Rob Orton 7b24afd223 cleanup invalid observer_alerts
fixes KNO-498

test plan
 - migrations should run

Change-Id: I727d9178d00bb88d5656def43df3d0417d3eadda
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/241840
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Davis Hyer <dhyer@instructure.com>
QA-Review: Davis Hyer <dhyer@instructure.com>
Product-Review: Davis Hyer <dhyer@instructure.com>
2020-07-09 16:21:38 +00:00
wdransfield 905df065e9 bump root_account_id migration timesamp
Change-Id: Ieddea338feca60316c8253ffc297df47530acdac
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/242205
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2020-07-08 22:32:07 +00:00
Ahmad Amireh cf98f71cd3 send upcoming assignment alert to unaware students
Fills in the stub for actually sending the notification to the selected
students.

Test Plan:
In a Rails console
 - Grab a student who has Due Date notifications set to Immediate (u)
 - Grab an assignment that the student is assigned, but has not
     submitted and is due in the future (a)
 - `s = Submission.find_by(user_id: u.id, assignment_id: a.id)`
 - `n = BroadcastPolicy.notification_finder.by_name('Upcoming Assignment Alert')`
 - `AssignmentUtil.alert_unaware_student(n, assignment: a, submission: s)`
 - Verify that a notification was sent

fixes VICE-445, VICE-446
flag  = smart_alerts

Change-Id: I56032152c8f9a0c4c2387d28420c419f990440d1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/240397
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Matthew Lemon <mlemon@instructure.com>
QA-Review: Matthew Lemon <mlemon@instructure.com>
Product-Review: Matthew Lemon <mlemon@instructure.com>
2020-07-07 19:31:53 +00:00