fixes CORE-3156
test plan
- specs should pass
Change-Id: I3e90d4c8ad9aa022c6fa2257c85d9b656e005010
Reviewed-on: https://gerrit.instructure.com/200364
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes PFS-13047
test plan
- check documentation for login importer
- import a login
- it should work
- user importer should work
Change-Id: I0e72aa75d1abd26b7b00a1d416401bc1cbc8732a
Reviewed-on: https://gerrit.instructure.com/198678
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
refs CORE-3093
test plan
- specs should pass
Change-Id: I1eb0345f9672e8eae842bfd0a3cf612d7b8552b7
Reviewed-on: https://gerrit.instructure.com/187495
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Change-Id: Id8eb68c773b7f867d07789dc05994a65985be4a3
Reviewed-on: https://gerrit.instructure.com/196410
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
and don't create parallel importers when calculating a diff
closes #CORE-3030
Change-Id: I98b07467575bea8950581034688deda5112ab907
Reviewed-on: https://gerrit.instructure.com/196205
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes CORE-3015
test plan
- run sis import under load
- it should process all of it
Change-Id: I43179d7a9073732a5195d47899bbc500b320fab3
Reviewed-on: https://gerrit.instructure.com/195636
Tested-by: Jenkins
Reviewed-by: Nathan Mills <nathanm@instructure.com>
QA-Review: Nathan Mills <nathanm@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
use the new cacheregister keys for a more reliable store
that should only be invalidated when an account_user
is created for the user
should improve performance of permission calls
refs #CORE-2851
Change-Id: Ia293c8e206a2fd83e7b6bc6e17d29d36c9cae87f
Reviewed-on: https://gerrit.instructure.com/192666
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
test plan:
* when creating a batch using diffing, the 'csv_attachments'
returned in the api should only list the files uploaded
and "diffed_csv_attachments" should return the files
created by the diffing process
closes #CORE-2914
Change-Id: Iabc4ef26e989329c9d1b334ac6dee38ccb3ad50e
Reviewed-on: https://gerrit.instructure.com/192578
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes CORE-2924
test plan
- delete enrollments via sis import
- it should update sis_batch_id
Change-Id: I42c1c881072123317b47c2f1411930f828e38653
Reviewed-on: https://gerrit.instructure.com/193028
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Tested-by: Jenkins
use the redis-based cache keys inside several calls that
rely only on a user's enrollments, group memberships, or
account_users
test plan:
* have redis configured as the cache store
* updating a user's enrollments, group memberhips,
or account admin associations should not result in
stale data being shown
refs #CORE-2851
Change-Id: I773a20eba40ff02a6aa37e35ffe345ba3554c4fd
Reviewed-on: https://gerrit.instructure.com/192271
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
fixes CORE-2890
test plan
- import a large sis import
- abort the batch early
- restore states soon after abort
- the remaining parallel jobs should clear out
Change-Id: I9ab681705e310bb10491a70de2a4332f9a958250
Reviewed-on: https://gerrit.instructure.com/191969
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
test plan:
* use the sis imports API to create a diffed sis batch
* refer to the API docs for 'diff_row_count_threshold'
to use the setting similarly to 'change_threshold' but
instead set an explicit row count and confirm that
diffing does not happen if the number of rows that
would be run in the diff exceeds the threshold
closes #CORE-2711
Change-Id: Idfc77d0aa64a82cf0665ffb5e3cc31f04d9e60e3
Reviewed-on: https://gerrit.instructure.com/189099
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
test plan:
* performa a sis csv import with a diff id
* use another sis csv import with the same diff id to
try to enroll a user multiple times in the same
course but with different roles using the "role_id" column
* it should enroll the user in all the roles specified
closes #CORE-2690
Change-Id: Idd1adc0c58afdd51ea7ca82d7cce0624567f3d11
Reviewed-on: https://gerrit.instructure.com/186678
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
test plan:
* use a sis csv import to create
a course,
a student and an observer user,
a user_observer link between the two users,
and an enrollment for the student
* make sure the observer was automatically enrolled
in the course along with the student
* use another sis csv to delete the student user
* the observer enrollment should be deleted
closes #ADMIN-2469
Change-Id: I54b237c8a727464f58505f7de380eea38a522a29
Reviewed-on: https://gerrit.instructure.com/184298
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
test plan:
* run the following in a rails console:
Account.default.tap do |a|
a.settings[default_notifications_disabled_for_observers] = true
a.save!
end
* use a sis batch import with user_observers.csv to create
an observation link for a student and observer
* as the observer, view the "Notifications" section of their
account settings
* they should all default to "Never"
closes #COMMS-1898
Change-Id: Iabf5bff42efa521bdad34ccf7de277cee0e1789c
Reviewed-on: https://gerrit.instructure.com/184022
Tested-by: Jenkins
Reviewed-by: Landon Gilbert-Bland <lbland@instructure.com>
Reviewed-by: Steven Burnett <sburnett@instructure.com>
QA-Review: Steven Burnett <sburnett@instructure.com>
Product-Review: Steven Burnett <sburnett@instructure.com>
test plan
- specs should pass
Change-Id: I08d1b932ecb06f04bd2d3c16662debd75a832868
Reviewed-on: https://gerrit.instructure.com/183059
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
test plan
- it should complete the last enrollment
- it should delete other enrollments
Change-Id: I7f09c2a2a52798ff9118a8765aa3ae76dc041e80
Reviewed-on: https://gerrit.instructure.com/181692
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
closes gh-1402
test plan
- Create a user with two logins
- have an enrollment tied to the sis id of one
- delete that pseudonym
- run a sis import enrollments referencing the
deleted sis id
- the enrollment should still be active
Change-Id: I350a998f53aae00662f2a133c17dd9596793ed6a
Reviewed-on: https://gerrit.instructure.com/178116
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
fixes CORE-2325
test plan
- run an import that has an empty column
- it should not error out with unhelpful error
Change-Id: Ieb2dd6186dd25178f5755f5b74b2d567ca25a70f
Reviewed-on: https://gerrit.instructure.com/177166
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
closes #CORE-2249
Change-Id: I4f857b84dcf0fccb43661c0d908b7786c871b88e
Reviewed-on: https://gerrit.instructure.com/175251
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Change-Id: I4786663805d277f028531d9c4a85a147427e7c66
Reviewed-on: https://gerrit.instructure.com/173987
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
fixes CORE-2200
test plan
- cause an import to fail
- no future parallel importers should process
Change-Id: I9d008434a39913037505e9cc4a64be299cfec0d1
Reviewed-on: https://gerrit.instructure.com/173645
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes CORE-2129
test plan
- specs should pass
Change-Id: Iae471745c1107a6af229dd347c3489ef6748c805
Reviewed-on: https://gerrit.instructure.com/171800
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
Change-Id: Ieb64f4f2cd5880bba1e815b6d4e2c28a45606e45
Reviewed-on: https://gerrit.instructure.com/171073
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
g/167863 will keep the default from getting stored in multicache
but it can still collide in the in-process cache
closes #CORE-1977
Change-Id: Ic29292ad1da69af9684862a934698ed73d001937
Reviewed-on: https://gerrit.instructure.com/167920
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
closes #CORE-1954
Change-Id: I1546a446230a8e4ab4b4502a4a6ab7d5b778aef6
Reviewed-on: https://gerrit.instructure.com/166568
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
closes GRADE-1487
Test plan:
- Specs pass
- Smoke test the following for an anonymous (or otherwise auditable)
assignment:
- Changing the due date for the whole assignment, a particular
section, or a particular student
- Adding or removing a student
- Updating student groups
- Updating grading periods
- Smoke test the above for a non-auditable assignment and make sure the
functionality works as expected
Change-Id: I31858a38059bc3ff73d91dd138ffa20cd8a76fac
Reviewed-on: https://gerrit.instructure.com/164776
Reviewed-by: Derek Bender <djbender@instructure.com>
Reviewed-by: Keith T. Garner <kgarner@instructure.com>
Tested-by: Jenkins
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: Keith T. Garner <kgarner@instructure.com>
fixes CORE-1905
test plan
- active sections should require a name
- deleted sections should not require a name
Change-Id: I50b25eee6755552bcb998582967874abe88ced1f
Reviewed-on: https://gerrit.instructure.com/164857
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
fixes CORE-1898
test plan
- have a course with start and end dates
- run sis import with that course and don’t
include the columns
- the dates should still be present
Change-Id: I7dcda15a2c75108ca98cfe44d59b3c3554fcf8eb
Reviewed-on: https://gerrit.instructure.com/164630
Reviewed-by: James Williams <jamesw@instructure.com>
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
- run completion update in transaction
- queue date/access recalculation job after other update in
sis importer
- also don't send a live event if nothing really updated
because it's silly
refs #CORE-1739
Change-Id: I0f0aaa0374c2dbee2cad426152da1cb16549bfc8
Reviewed-on: https://gerrit.instructure.com/164418
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
- preload attachments
- don't re-save the attachment even if the downloaded
filename doesn't match the one saved in the db
(the code meant to work with the xml file hooks was
too overzealous)
- don't reload items or make progress saves if we're
running the batch immediately
- don't make a bunch of error queries if we don't have to
test plan:
* big sis imports should work
* small sis imports should work (and faster)
closes #CORE-1801
Change-Id: I4291e6b4af9874fa3c0b8a012400ba428381f09c
Reviewed-on: https://gerrit.instructure.com/162823
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
test plan:
* delete an enrollment via sis csv
* the batch should include the deleted enrollment
in the enrollments count
closes #CORE-1761
Change-Id: If98cdee6920b782878f4f4e3777036f0e93d5719
Reviewed-on: https://gerrit.instructure.com/161834
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
refs CORE-1738
test plan
- group membership importer should work
Change-Id: I3678dfa3501ccb48a0de6db5fa32dd50a9c2a1c7
Reviewed-on: https://gerrit.instructure.com/161309
Reviewed-by: James Williams <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
fixes CORE-1738
test plan
- have a deleted group membership in a group
- have an active group membership in a group
- import an active group membership via sis csv
- it should not fail
Change-Id: Ide644d7f83601c603287cbcdb376f27a4e6ce9d4
Reviewed-on: https://gerrit.instructure.com/161275
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
after filtering out passwords of course
test plan:
* with the sis imports refactor feature flag on,
upload some zips and csvs to sis imports,
including some users.csv files with passwords
* view the sis imports API index and show
endpoints
* should be able to use the urls in the new
"csv_attachments" attribute to download versions
of the imported files without passwords
closes #CORE-1655
Change-Id: I31e34d42f4abf2597efd6066dabea6f230632855
Reviewed-on: https://gerrit.instructure.com/158899
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
Change-Id: Ifef519776e4358204dd6625acd6cd9f841cd7fcb
Reviewed-on: https://gerrit.instructure.com/152331
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
fixes CORE-1729
test plan
- sis import refactor should be on
Change-Id: I9d6ec6b9e548b7ded62d940a353ca1930ead4cc4
Reviewed-on: https://gerrit.instructure.com/160998
Reviewed-by: James Williams <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
when a failure gets caught by the job processor
it won't have the old shard active anymore
refs #CORE-1676
Change-Id: I496219c330cf0afc9fd898eb2e7c5b20361f25c1
Reviewed-on: https://gerrit.instructure.com/159242
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
fixes gh-1325
test plan
- checkout canvas-lms and this commit with
no plugins
- run an enrollment sis import
- it should pass
Change-Id: I9b93da6dec381ba19d103c468a712ec18db6b43e
Reviewed-on: https://gerrit.instructure.com/158443
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
test plan:
* turn on the sis import refactor feature flag
* importing a small sis import (default 50 rows or less)
should show the number of successfully imported rows
for each type on the sis import page
closes #CORE-1634
Change-Id: Ifdbe98c8bc9252552e7280d3df0764503a388396
Reviewed-on: https://gerrit.instructure.com/157791
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
using ids from now on will speed up the job deserialization
(and the code should still work with existing jobs too)
also fix a problem with error handling when the error
is just a string
Change-Id: I6315e523c6f02050ac3d09b67e693ca616636502
Reviewed-on: https://gerrit.instructure.com/154812
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
test plan:
* have the sis import refactored feature flag on
* use
Setting.set('sis_batch_parallelism_count_threshold', 'X')
in a rails console to set the threshold where sis batches
use the parallel jobs when the batch has enough rows
* stop job processing
* queue multiple sis batches some with fewer rows
and others with more (above the threshold)
* only one job should be queued in /jobs
* restart job processing
* it should run all the batches
closes #CORE-1485
Change-Id: I2f8583b37273f5e8f39cb668fc690fa617b894ca
Reviewed-on: https://gerrit.instructure.com/152912
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
QA-Review: Rohan Cheeniyil <rcheeniyil@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
Change-Id: Iead855798aeb864a86cb35e8fb17a497b633cfc8
Reviewed-on: https://gerrit.instructure.com/153823
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
refs #CORE-1483
Change-Id: Ib30a5b299fb01ca1df70948d6b1fead45b8e7b18
Reviewed-on: https://gerrit.instructure.com/152620
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
refs #CORE-1467
Change-Id: I57d2f3315e7c908bfb7f5944b650e4153a965908
Reviewed-on: https://gerrit.instructure.com/151863
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Rob Orton