if the sis import fails to find the sis_batch on
update progress it will continue and update the
progress on next run.
fixes CNVS-34250
test plan
- run a sis import
- it should run
Change-Id: Ie4541b250f375bcb5fedfa6a1ab9afee15233169
Reviewed-on: https://gerrit.instructure.com/99230
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
test plan
- run enrollment import
- it should work
- if unique index error happens it should continue
Change-Id: If128dbe34b2e41a8139f5afb8d87f9d424dd024a
Reviewed-on: https://gerrit.instructure.com/99088
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes CNVS-33458
test plan
- specs should pass
Change-Id: Ia63bc50ae91e5742a3a3684e500f470a7ba672fe
Reviewed-on: https://gerrit.instructure.com/95770
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
fixes CNVS-32336
test plan
- run an invalid sis import with invalid login_id
- it should display warnings and not an error on
/sis_imports page
Change-Id: I708ee42c359a154bad6574417daac44839317177
Reviewed-on: https://gerrit.instructure.com/91767
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Benjamin Christian Nelson <bcnelson@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
closes CNVS-31154
don't touch users within transaction, that way
we don't deadlock when this is running at the
same time as another importer that tries to
touch_all for some users in a different course
and overlaps.
Change-Id: I1ccca396d7bcf1e984c5854108c5acb5a4875d63
Reviewed-on: https://gerrit.instructure.com/88410
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
caused by sis user deletion and course state changes
also DRY up the enrollment recalculation a little
closes #CNVS-31060
Change-Id: Ife8b2a95223c423f2faec79f3f9d5af5377ce51e
Reviewed-on: https://gerrit.instructure.com/88307
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
test plan:
- add a "course_format" column to courses.csv
- put "on_campus", "online", and "blended" in rows
- perform a SIS import
- ensure in course settings that the setting was
imported properly
- ensure the setting is updated after a re-import
closes CNVS-30561
Change-Id: I0153c91c897cdffeb9021b65127006961989494e
Reviewed-on: https://gerrit.instructure.com/85967
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Heath Hales <hhales@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
test plan:
- create a user with a SIS ID
- add this user as an admin in an account
- delete the user via SIS import
- ensure the user is no longer an admin in the account
- also test when the user is a subaccount admin
and ensure the subaccount membership is deleted
when the user is deleted
- ensure that an admin in multiple root accounts/shards
is only deleted from one of these accounts via a
SIS import (the domain the SIS import was run from)
fixes CNVS-30275
Change-Id: I288e10219fe39905fd271d02df1dc0aeb9814e76
Reviewed-on: https://gerrit.instructure.com/84852
Tested-by: Jenkins
Reviewed-by: Alex Boyd <aboyd@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
fixes SIS-2121
test plan
- sis imports should work when passing closed or
completed
- specs should pass
Change-Id: If7d976a774448aaa8db3d748c46bc485cfe7bf7c
Reviewed-on: https://gerrit.instructure.com/82120
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes CNVS-29759
test plan
- processing a file full of ",,,,," should
process a lot faster.
- it should not include them in the counts for
the import
Change-Id: I02cecc7e132f39b9a241b891f7484c0691130c54
Reviewed-on: https://gerrit.instructure.com/81344
Tested-by: Jenkins
Reviewed-by: Brent Burgoyne <bburgoyne@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
closes CNVS-29218
test plan
- test user_observer imports
- test creation, deletion, diffing mode
- check that deleting user_observers also removes
observer enrollments for the observer/student
- generate docs and check /doc/api/file.sis_csv.html
Change-Id: I1c6a5a5e231ea4deb1bd9d214651b2a851e56887
Reviewed-on: https://gerrit.instructure.com/79088
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
Reviewed-by: John Corrigan <jcorrigan@instructure.com>
refs CNVS-29218
test plan
- specs should pass
Change-Id: I44731a0fb2b41b0f2f5390ce2e9d9030a7a9be05
Reviewed-on: https://gerrit.instructure.com/79191
Tested-by: Jenkins
Reviewed-by: John Corrigan <jcorrigan@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
shouldn't log "Published" event when it isn't publishing
closes #CNVS-27689
Change-Id: Ie8580e29e68c228ebe749513a20e3e670ee323d1
Reviewed-on: https://gerrit.instructure.com/73486
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
fixes CNVS-27396
test plan
- run an enrollment import that has a course sis
id and a section sis id that belongs to a
different course than the one provided
- the warning should say the sis id for each the
course, section, and user
Change-Id: I4c7ae135465da0a534fda58e0e013d5c5a85c8fc
Reviewed-on: https://gerrit.instructure.com/72647
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Rob Orton <rob@instructure.com>
In order to facilitate better and more efficient debugging, this commit
improves the quality of the error messages reported by SIS imports. When
a user does not exist in users.csv, and enrollments in enrollments.csv
reference that user, the message "User XXX didn't exist for enrollment"
is not particularly useful. This commit adds course and section IDs to
the message to make it easier to search for broken rows.
Test plan:
* Create or change an existing enrollments.csv file so that it
references a user that doesn't exist.
* Run the CSV through the SIS importer.
* Check the error messages for the last import; the new error message
should show up instead of the old one.
Change-Id: Ie3ea4d95c6877ef8c5b27cee9d530a6fdf0e6e6e
Fixes: SIS-1733
Reviewed-on: https://gerrit.instructure.com/71116
Tested-by: Jenkins
Reviewed-by: Ken Romney <kromney@instructure.com>
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
QA-Review: Steven Shepherd <sshepherd@instructure.com>
Product-Review: Ken Romney <kromney@instructure.com>
Test Plan:
Create a user.csv file to import via SIS Imports with a row
containing an invalid login_id.
Note that the message is not "unique_id is invalid"
Note that the message contains contextual login_id and
sis_id / user_id information
Fixes: SIS-1732
Change-Id: I16f47d57f4dbe2550d979e6f6552d164a8709631
Reviewed-on: https://gerrit.instructure.com/71056
Product-Review: Linda Feng <lfeng@instructure.com>
Tested-by: Jenkins
Reviewed-by: Max Stahl <mstahl@instructure.com>
Reviewed-by: Nick Houle <nhoule@instructure.com>
QA-Review: Kausty Saxena <kausty@instructure.com>
In order to facillitate better debugging and customer support efforts,
the error messages given for enrollments that had a valid user but not a
valid course or section id were improved. The error message now includes
the relevant course ID, section ID, and user ID, in a convenient format,
in addition to the old message.
Example error message (old):
Neither course 1 or section 2 existed for user enrollment
Example error message (new):
Neither course nor section existed for user enrollment \
(course_id=1, section_id=2, user_id=3)
Test plan:
* Create CSVs for SIS import
* Create `enrollments.csv` with valid enrollments for actual users
* Delete some lines from `courses.csv` and `sections.csv` to create
conflicts
* In the "last import" view, the error log should contain the more
detailed error message.
Fixes: SIS-1734
Change-Id: I52b82a7a83f7cf7df4a5845f6d3fc82b9623e47e
Reviewed-on: https://gerrit.instructure.com/71267
Tested-by: Jenkins
Reviewed-by: Ken Romney <kromney@instructure.com>
Product-Review: Linda Feng <lfeng@instructure.com>
QA-Review: Kausty Saxena <kausty@instructure.com>
refs #CNVS-26056
Change-Id: Ie5e8ce8e693b1b03aefc6dd56af5f4901e27a5a2
Reviewed-on: https://gerrit.instructure.com/70478
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
test plan:
* conclude an enrollment through sis csv import
* it should have a completed date on the user's course
profile page
closes #SIS-1655
Change-Id: Iba3f4d579e293df6f68964027e7b5a1f5b51d90f
Reviewed-on: https://gerrit.instructure.com/69740
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
e.g. use .seconds.ago instead of .ago
refs #CNVS-26056
Change-Id: I5af8541116623a8fc8b49682b0829a065aba59c8
Reviewed-on: https://gerrit.instructure.com/69339
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
fixes CNVS-25747
test plan
- enrollment sis import should work
Change-Id: Ifaecd8651d62a9df4e6695965f9f9747c6a5f845
Reviewed-on: https://gerrit.instructure.com/68609
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
fixes CNVS-23014
test plan
- run enrollment sis import
- it should work like before
Change-Id: I16f6f822ddbcbf1403268b399e68dca7c1f05be8
Reviewed-on: https://gerrit.instructure.com/68198
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
test plan:
* run a sis import that changes a user but does not
update their email address
* the imported items count should include a User
closes #CNVS-14648
Change-Id: I67dfdf167c8eed77d3b4839fed5dbf905bcf513a
Reviewed-on: https://gerrit.instructure.com/65222
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Tested-by: Jenkins
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
In testing SIS integrations we encountered a situation where seed data
from a partner had 1771 users with the same email address. Near the end
of the import process this lead to executing 5310 separate queries (3 x
1770) to determine how to handle the seemingly duplicated
CommunicationChannels.
The first query, existence of a user for a communication channel is
redundant because database constraints enforce that the user record
exist and that the CommunicationChannel user_id field not be NULL. This
check was removed.
The second query, existence of active pseudonyms for the other
communication channel's user, can be combined for all relevant users
into a single count query with a grouping by the user_id.
The third query, similar to the second with the addition of checking
the root account and presence of a SIS id, is similarly combined.
These three changes yield a 10x performance improvement in the mentioned
pathological case.
Test Plan:
- Run a user import with a repeated email address and ensure the system
ends with all communication channels in the correct state.
Change-Id: I5695cdf342249780d37780f61d56f8257f3f8452
Reviewed-on: https://gerrit.instructure.com/65262
Reviewed-by: Brad Humphrey <brad@instructure.com>
QA-Review: Kausty Saxena <kausty@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
Tested-by: Tyler Pickett <tpickett@instructure.com>
refs CNVS-22801
Test plan:
* Create a csv file with the following contents
user_id,login_id,first_name,last_name,email,status
user_1,user1,User,Uno,BAD_EMAIL_ADDRESS,active
* Try to import this csv file
* Ensure that the import succeeds (i.e. the user is created)
* Ensure there is an import warning due to "BAD_EMAIL_ADDRESS"
not being a valid email address
Change-Id: Iac85a894bbbbdd69cd4f88f6f893975f6c6436c5
Reviewed-on: https://gerrit.instructure.com/63990
Reviewed-by: Mark Severson <markse@instructure.com>
Tested-by: Jenkins
QA-Review: Adrian Russell <arussell@instructure.com>
Product-Review: Linda Feng <lfeng@instructure.com>
fixes CNVS-12892
test plan
- enroll a user in a course
- add them in a group in the course
- delete user through sis
- the group_membership should also be deleted
Change-Id: I86065eaaebb271594f16f3b60a586f42c897158c
Reviewed-on: https://gerrit.instructure.com/64854
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: August Thornton <august@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
Progress is stored as an integer but we were sending a float here, this
caused large imports to report 100% progress when they could have been
as little as 99.5% complete. For small batches this isn't an issue but
for very large customers this 1/2 percent could be many minutes to
hours.
Fixes: SIS-1403
Change-Id: I9fdcd3133e26a860c85a239652127184c179eab2
Reviewed-on: https://gerrit.instructure.com/62518
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
test plan:
* create a course through a sis import
* toggle the "users can only participate between these dates"
setting
* should be able to update course dates with sis import
closes #CNVS-22504
Change-Id: I620109dd5ed146082ac3141b37024db43617d5bd
Reviewed-on: https://gerrit.instructure.com/62319
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Charles Kimball <ckimball@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
test plan:
* create an auth provider
* do an sis import without auth provider; it should still work
* specify the auth provider by name in the import; it should work
* specify the auth provider by id in the import; it should work
Change-Id: I71f381a1bc140c3992cac6eb45bb077e5bd26baa
Reviewed-on: https://gerrit.instructure.com/59558
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs #CNVS-21596
Change-Id: I5dedaab90a2abe6bf288ff30401c9b31629b45b2
Reviewed-on: https://gerrit.instructure.com/59220
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
test plan:
* import a sis csv file longer than 4096 bytes with utf8 characters
split at the 4096 byte point
* should not fail with "Invalid utf-8 error"
closes #CNVS-20678
Change-Id: I642f24391bfaed48e16130989710056664ab4fd5
Reviewed-on: https://gerrit.instructure.com/56918
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Jahnavi Yetukuri <jyetukuri@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
fixes CNVS-19817
test plan
- make a custom observer role
- enroll a user to the role with an
associated_user_id
- it should work and have the associated_user
Change-Id: I4efe6c88271f0c1001eb5f6200b0c72cb133840e
Reviewed-on: https://gerrit.instructure.com/54513
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
closes CNVS-20176
in several places we check the postgresql version to see
if we can use the NO KEY UPDATE lock style. This abstracts
those checks into one place in our active record initializer.
TEST PLAN:
1) make sure course enrollments still works, enrolling in a course
makes use of this directly.
2) make sure importing a CSV for SIS still works.
Change-Id: Ifc3fe364058e22c75b4fb035695da0a2e5cf066d
Reviewed-on: https://gerrit.instructure.com/53861
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
test plan:
* import the sis .zip package referenced in the ticket
* should have a warning
* should not create an active enrollment in the new course
closes #CNVS-19829
Change-Id: I1f1aa23da7a7a8aab09026bc974945f8f3fc5743
Reviewed-on: https://gerrit.instructure.com/52576
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Tested-by: Jenkins
QA-Review: Jahnavi Yetukuri <jyetukuri@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
closes CNVS-6016
No more error reports! (soon)
this commit builds up sentry integration through the new
Canvas::Errors module, along with other things that need
to happen on every exception. ErrorReports
should now get pushed towards just being used for representing
a complaint a user filed via the get help form.
I fixed about half the things that got linted as well
while I was in here, but because this touches to much
I fear divergence from tackling too many (I think we
can safely say it's "better than we found it")
I left a lot of the infrastructure for error reports in place
until other commits for plugins can be merged
TEST PLAN:
1) setup your raven.yml config file with the dsn for our
sentry install
2) force an error to happen in a request response cycle.
3) see the error in sentry
4) force an error to happen in a job
5) see the error in sentry
6) statsd increments shoudl still fire
7) for the moment, an error report should still get created.
Change-Id: I5a9dc7214598f8d5083451fd15f0423f8f939034
Reviewed-on: https://gerrit.instructure.com/51621
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins
QA-Review: August Thornton <august@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
* don't check pause vars after every progress update
* base throttling interval _after_ the progress update completes,
in case it takes long enough on its own
Change-Id: I8b7046a9c89edcbb27296076c75527e4ce51a6ef
Reviewed-on: https://gerrit.instructure.com/49773
Tested-by: Jenkins
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
This opt-in preprocessing pass does a diff on the current SIS Import against
the last successful import tagged with the same "data set identifier",
and generates a new zip of CSVs containing only the diff between the
two. It then continues with the import as normal, applying that diff'd
zip, greatly speeding up large imports where most data doesn't change.
closes CNVS-18432
test plan:
Exercise the new diffing options using the diffing_data_set_identifier
and diffing_remaster_data_set api options. See the new API
documentation for details and expected limitations. Edge cases include:
- zips that contain multiple CSVs for the same data type (won't diff)
- failed SIS imports (shouldn't diff against these)
- remasters should skip doing the diff, but should be diffed against on
the next import
We don't expose any functionality for downloading SIS data that has been
uploaded, but you can grab it from the rails console:
> SisBatch.find(<batch id>).attachment.authenticated_s3_url
and
> SisBatch.find(<batch id>).generated_diff.authenticated_s3_url
Change-Id: I40d5e5ca8c376cecd685f4d06012f11bac021599
Reviewed-on: https://gerrit.instructure.com/48428
Tested-by: Jenkins
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Tyler Pickett <tpickett+gerrit@instructure.com>
Product-Review: Linda Feng <lfeng@instructure.com>
test plan:
* run an sis export (account report)
* note the presence of role_ids on enrollment.csv's
* should be able to specify a role_id in the column
and import for a specific custom role
refs #CNVS-18343
Change-Id: I4f6232b872ba5aa9fc311d5506f80e47af01a504
Reviewed-on: https://gerrit.instructure.com/48568
Tested-by: Jenkins
QA-Review: Clare Strong <clare@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
refs CNVS-17778
also put stuff into the cache even if it hasn't changed
Change-Id: Ifb9a857b2d38303208a20e5a6993349525c56d1b
Reviewed-on: https://gerrit.instructure.com/47215
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
fixes CNVS-17979
test plan
- create an enrollment csv with an invalid
associated_user_id for an observer
- the import should not fail
- the import should give warning about the bad data
Change-Id: Iaecd177b8be24efdece1db21890beae5d206cdc2
Reviewed-on: https://gerrit.instructure.com/47134
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: August Thornton <august@instructure.com>
fixes CNVS-16542
when a section is cross listed to a course and the
course is deleted the section is treated as
deleted and the section will be uncrosslisted on
an sis import when state of the section is set to
active
test plan
- create a section in course a through sis
- crosslist the section into course b
- delete course b
- import the section again into course a or
course c, and it should move and be active
Change-Id: I84d8f3a6d1c7cfa8d2ae3f2d2a0f890d2cccab49
Reviewed-on: https://gerrit.instructure.com/46997
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
refs CNVS-17778
test plan:
* import a batch with duplicate lines
* it should not fail
Change-Id: I568c3ef8f7173a9d4c287d362e0415af29679647
Reviewed-on: https://gerrit.instructure.com/47150
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Change-Id: I167620165171b6fd0d41e69590e1f3225ac2fe2c
Reviewed-on: https://gerrit.instructure.com/45690
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
fixes SIS-661
Test plan:
- confirm integration_id is now being loaded from user csv
- specs pass
Change-Id: Ie47c1bf9420871b1a017ebc0352d968e1564dda7
Reviewed-on: https://gerrit.instructure.com/45121
Reviewed-by: Ken Romney <kromney@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cosme Salazar <cosme@instructure.com>
test plan:
* enroll a student and an observer through an sis import
* delete the student enrollment through sis import
* delete the observer enrollment through sis import
* should properly delete the observer enrollment
closes #CNVS-1729
Change-Id: Ief3135f99549bdafa8f9068aeaca7c5e91fe8152
Reviewed-on: https://gerrit.instructure.com/44599
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Sean Lewis <slewis@instructure.com>
Change-Id: I4beb68a93f6eb4a613e63d12ec79dbca179083ed
Reviewed-on: https://gerrit.instructure.com/44447
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
fixes SIS-627
Test Plan:
- To allow Canvas to be translated you must start canvas with
these options:
-- RAILS_LOAD_ALL_LOCALES=true USE_OPTIMIZED_JS=true rails s
- Confirm all translations in these screenshots are translated
-- http://screencast.com/t/DUuU85s0
-- http://screencast.com/t/HDawR9681i
-- http://screencast.com/t/QOVas4gfd
NOTE:
- If you're using the LOLCALIZE=true option to test translations
months, days, etc. will not be translated by LOLCALIZE but will be
translated in another language.
- If you're using Spanish or French as the language be aware that
some of the short hand months are the same as english becasue
they are related closely to english.
Change-Id: Ia14a3ec4408f40c9bcdd309c7088c5165519a1f9
Reviewed-on: https://gerrit.instructure.com/43983
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jayce Higgins <jhiggins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Nick Houle <nhoule@instructure.com>
refactor everything that used to use strings for roles
to use actual role_ids
the apis should be backwards compatible so we don't need
to update (most of) the UI's right away in this commit
test plan:
* regression tests for permissions, role overrides,
alerts (for account roles), enrolling users,
adding account admins, etc.
refs #CNVS-15481
Change-Id: Id57fd3104c5c518b6fbf180609950dcddcdd474d
Reviewed-on: https://gerrit.instructure.com/41208
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Steven Shepherd <sshepherd@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
test plan:
* create a student
* as a user with a valid login,
observe the student permanently using the
observees UI ("/profile/observees")
* enroll the student in a course through
an sis import
* should create an observer enrollment
for the observer as expected
closes #CNVS-14943
Change-Id: I087190b3672b51d2b2f91551ea3a0a424ab89091
Reviewed-on: https://gerrit.instructure.com/42205
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Jahnavi Yetukuri <jyetukuri@instructure.com>
This email only went to site admins, and isn't useful.
This commit stops new notifications being created, a subsequent commit
will remove the no longer used code and data.
refs CNVS-15219
Change-Id: Ibd8c88c7310cf3dcfb06cf0c782f2b3cf571d843
Reviewed-on: https://gerrit.instructure.com/40375
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
the marshalling to/from the child serialized delayed job isn't including
@last_progress_update, causing errors comparing a time to nil.
refs CNVS-14640
Change-Id: I6636335202921c0210edce2ebebd4eef4ec22dac
Reviewed-on: https://gerrit.instructure.com/39358
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
Rather than based on a # of rows processed metric
fixes CNVS-14640
test plan: run a large-ish SIS batch, and it should succeed as you'd
expect. the progress in the UI/API should still update as it runs (and
when it completes), but maybe not quite as often.
Change-Id: I44e30c1bbd1cd70ade7633c48389bf6bf0d04773
Reviewed-on: https://gerrit.instructure.com/39281
Reviewed-by: Zach Wily <zach@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
Tested-by: Brian Palmer <brianp@instructure.com>
extract core unzip functionality into canvas_unzip gem, and put
security logic there. use this gem instead of shelling out to
`unzip` (which does not have the option to skip symlinks).
test plan:
1. import 'evil_course_2.imscc' from CNVS-14338
* there should be an import warning
* you should get a blank syllabus body and
definitely not see sensitive system data
2. import 'evil_sis_import.zip' from CNVS-14346
* a file called '/tmp/pwn3d' should not have been
created on your app server
3. sanity check the parts of canvas that unzip things:
* course copy
* course import
* zip content imports via the API
* zip file uploads from files page
* assignment submission comments download/upload
* sis imports
fixes CNVS-14338
fixes CNVS-14346
Change-Id: I38fa141653eb7bc483e99a28a135831b8cb3b2a6
Reviewed-on: https://gerrit.instructure.com/37959
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
fixes: CNVS-13302
Fixes the sis imports to log the sis batch correctly and to display the sis
batch on the details modal in the auditing ui.
Test Plan:
- Do a SIS import for a course.
- Open the Course Activity Audit logging and the sis batch id for the course
should be shown in the dialog.
- If the source for an event is not SIS the sis batch field will not be shown.
Change-Id: Ib93a992dac83b2ec480d888e5d4686b808bcda8f
Reviewed-on: https://gerrit.instructure.com/35475
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Nick Cloward <ncloward@instructure.com>
Add full & sortable name fields to SIS User Import
Change-Id: Ib8617572802ca69913b422d22301299ab75c211f
Reviewed-on: https://gerrit.instructure.com/34548
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
fixes CNVS-12963
test plan:
* set up a trust from account A to account B
* enable sis imports for account A and account B
* add a user to account B with an SIS ID
* add a course to account A with an SIS ID
* do an sis import to account A for enrollments as user
that is an admin in both account A and account B;
specify the course from account A as the target,
the user from account B, and add another column
root_account with account B's domain
* it should succeed, and enroll the user from account B
* unenroll the user manually
* do the import as a user that's an admin in account A,
but not account B - it should fail citing permissions
* remove the trust link from A to B
* do the import as the user that's an admin in both;
it should fail citing no usable login
Change-Id: Ie5b7b71bfe563da9c49d3aa2321586994634ccb5
Reviewed-on: https://gerrit.instructure.com/32133
QA-Review: August Thornton <august@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
fixes: CNVS-347
Adds state change logging activity for courses. This change also adds an event
source field to all the course activity events. The options for this is api,
manual, and sis.
Note: This also adds a fix for the event stream record object to force request_id to
be a string. Event stream will try to insert an integer into a text column which
Cassandra will complain about and throw an error.
Test Plan:
- Test for each type of Event:
* Concluded
* Unconcluded
* Published
* Deleted
* Restored
* Updated
* Created
* Copied To
* Copied From
- Test each type of event and each source. Note that some events can only
have certain sources. Also make sure SIS migrations show that the source
is type SIS.
Change-Id: Ic2308e1d10e6fc1cc6437221d7842eaf1a596f46
Reviewed-on: https://gerrit.instructure.com/30343
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Nick Cloward <ncloward@instructure.com>
Both fields are optional. If absent, the user's name and sortable_name
will be set in the same manner as before.
The full_name field takes precedence over first_name and last_name. The
sortable_name field takes precedence over the inferred value.
The display name (short_name) field is optional. If absent, short_name
will behave as before. If present, it will explicitly set short_name to
the given value, unless it's a stuck SIS field of course.
Previously, if first and last names were not given, the user's name will
be a single space. The behavior is the same now, except when short_name
is present. In that case, an error will be thrown.
fixes: CNVS-11546
Adds a user relation to sis batches so we can relate a sis batch job back to the
user who initiated it.
Test Plan:
- Initiate SIS Imports.
- The associated sis batch records user_id column should have the correct value.
* SIS Imports should not be broken.
Change-Id: Ic5d0807af4d945a150a0268776e51c6db96b29c6
Reviewed-on: https://gerrit.instructure.com/31172
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Nick Cloward <ncloward@instructure.com>
refs CNVS-11680
test plan
- on mysql run user import
- should not fail
Change-Id: I3c6ccc99aae493ba87fc5c2ebaf8e789dfca0338
Reviewed-on: https://gerrit.instructure.com/31543
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
fixes CNVS-11680
test plan
- import user.csv with a different case than
existing user's unique_id
- no import warning errors should be shown
Change-Id: I64a3a84f6874ec88c4f28bc046f80cad9219fc43
Reviewed-on: https://gerrit.instructure.com/31453
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Eric Adams <eadams@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
refs CNVS-9522
replace our hokey ActiveRecord::Base.suspend_callbacks implementation
with something that integrates nicely with ActiveSupport::Callbacks in
general, as a gem.
include it into ActiveRecord::Base and use it to implement
save_without_callbacks for rails3 (rails2 still needs to do it the way
we had it before, because the notifications callback isn't handled
through ActiveSupport::Callbacks).
test-plan:
- make sure ActiveRecord callbacks still run in most cases (e.g. after
creating a course through the UI, it should have a value in the uuid
column in the database)
- make sure callbacks don't run while suspended (e.g. have a user
enrolled in two courses. call user.delete_enrollments. both
enrollments should be deleted, but only one delayed job for due date
caching should be created)
Change-Id: Iecd1cc0346e218261677d16ee44bae9cb9149ddd
Reviewed-on: https://gerrit.instructure.com/29809
Reviewed-by: Anthus Williams <awilliams@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
fixes SIS-162
TestPlan
----------
- import user.csv with a different case than existing user' unique_id
- no import warning errors should be shown
Change-Id: I653bcc7c2ace9953fe74ada0c0ab7da32c65662c
Reviewed-on: https://gerrit.instructure.com/30151
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Duane Johnson <duane@instructure.com>
Product-Review: Duane Johnson <duane@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Eric Adams <eadams@instructure.com>
refs CNVS-9522
in rails3 ActiveRecord::Base.skip_callback is a real thing that doesn't
do what this does. because of load order, ours happens, but it would be
nice to leave the other available.
to simplify the switch, both suspend_callback and suspend_callbacks are
kept as renames, but suspend_callback should not be used, preferring
suspend_callbacks. suspend_callback will go away in the near future.
skip_callback is temporarily added back in as an alias to
suspend_callback to allow time for fixing a plugin that uses
skip_callback. it will be removed as soon as that plugin is updated.
Change-Id: Iaefd16dde3b6ce575780cb8f721dc826258eef4e
Reviewed-on: https://gerrit.instructure.com/29808
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Nick Cloward <ncloward@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
in rails 3, ActionController::TestUploadedFile has
been moved to Rack::Test::UploadedFile. This commit
simply updates uses of TestUploadedFile to work
with this new structure
Change-Id: Ib31159c635f033a13908608dffeea88c8f719086
Reviewed-on: https://gerrit.instructure.com/28234
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Anthus Williams <awilliams@instructure.com>
QA-Review: Anthus Williams <awilliams@instructure.com>
Test Plan:
- Import a CSV with integration_id column in any
of: account, course, section, term, user
- The integration_id should be available in the
console for objects imported in this way (e.g.
user = User.find(123); user.integration_id)
Fixes SIS-108
Change-Id: Ie7f0765c2ec7222d8c7ba7dfbf78c858812e62d3
Reviewed-on: https://gerrit.instructure.com/27672
Tested-by: Jenkins <jenkins@instructure.com>
Tested-by: Ken Romney <kromney@instructure.com>
Reviewed-by: Ken Romney <kromney@instructure.com>
QA-Review: Ken Romney <kromney@instructure.com>
Product-Review: Ken Romney <kromney@instructure.com>
test plan: zip functionality should continue to work, including:
* course exports and imports
* zip file uploads
* zip submissions
also, the selenium dependency on rubyzip 1.0.0 is resolved
and we're making progress toward > 4GB exports
Change-Id: I58c5b0644b1e7fbb289821c9c0901f00750988de
Reviewed-on: https://gerrit.instructure.com/25474
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bryan Madsen <bryan@instructure.com>
Product-Review: Bryan Madsen <bryan@instructure.com>
QA-Review: Matt Fairbourn <mfairbourn@instructure.com>
deleting a user could cause thousands of update-cached-due-dates
jobs to be enqueued (one per assignment per enrollment; in the
referenced ticket, the test student had 13 enrollments and there
were 88 assignments, yielding 1,144 jobs, causing the request
to time out).
fix this by
(1) using a batch job to update all assignments in the course,
instead of a job per assignment
(2) when deleting a user with multiple enrollments in a course,
only run the update cached due date job once per course
test plan:
0. run in "production" configuration (i.e., in QA portal)
1. have a course with 10 sections and 100 assignments.
I suggest creating these in the console, e.g.
course = course.create! name: "CNVS-8068"
10.times { |x| course.course_sections.create! name: "section #{x}" }
100.times { |x| course.assignments.create! name: "assignment #{x}" }
2. enter student view mode in this course
3. reset the student view student
4. verify 1000 jobs were not created (and the request doesn't time out)
fixes CNVS-8068
Change-Id: If00dd3197b70df42b0f9ec18303b02594861f69f
Reviewed-on: https://gerrit.instructure.com/25160
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Matt Fairbourn <mfairbourn@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
fixes CNVS-8071
test plan:
* create a course and a section with an SIS ID
* create a an assignment, and override it's date for that section
* enroll two users via SIS into the course; one with no section
(i.e. the default section), and one into the section with the override
* confirm that the users show appropriate due dates for the assignment
Change-Id: I0e2b64d1d7095d370ba981b3b0f3ec4d1742f15b
Reviewed-on: https://gerrit.instructure.com/24254
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
fixes CNVS-7266
test plan:
* create an SIS user with a login id, then delete them
* create a non-SIS user with the same login id
* use an SIS import to resurrect the original user
* the SIS import should succeed, but warn that it skipped the user
cause someone else had his login information
Change-Id: I2102107fefb1d20fae6046ebe3a7e0c03416dcc9
Reviewed-on: https://gerrit.instructure.com/22867
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
- update the Gemfile to be 1.9 only, and raise an exception on wrong
ruby version
- remove RUBY_VERSION checks, replacing with the applicable code
- remove the FasterCSV compatibility shim, just use CSV now
test plan: trying to bundle install on ruby 1.8 or 2.0 should raise an
exception, specs should pass, canvas should work as normal on 1.9
Change-Id: I49088e9d227c59c6d5d5acb417c2df971129474a
Reviewed-on: https://gerrit.instructure.com/19806
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
There is also a SIS::BaseImporter in the outer module/namespace, which
makes having the same class name in a nested module really error prone.
Change-Id: Ia61f87088748212f855577570a28871c06fe87d9
Reviewed-on: https://gerrit.instructure.com/19292
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Duane Johnson <duane@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
We were inconsistent about whether to require
'lib/sis/csv/base_importer' or 'sis/csv/base_importer', causing warnings
in the logs about already initialized constants (not to mention
evaluating the same file twice, though that seems to be ok in this
case).
rather than fix the requires to be consistent, i've just removed them,
as rails autoloading works for all but the sis/common require (doesn't
work for sis/common because that defines things other than a SIS::Common
module)
test plan: doing SIS imports should still work as normal, and you
shouldn't see any warnings in the logs about already defined constant
PARSE_ARGS
Change-Id: I787ee7486d967201bfb360bc95e61e48b385c449
Reviewed-on: https://gerrit.instructure.com/19190
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
When a GroupMembership is updated, don't check to make sure the user's
group is within the section, as this could impact SIS imports. We
only care about validating homogeneity when creating a new
GroupMembership.
Test Plan:
- create a batch import in which a user is enrolled in a class,
and is also in a group within that enrollment
- add something else to the batch (to test that the rest of the
batch is imported even when the error, below, occurs)
- import the batch
- mark the enrollment as deleted
- re-import the batch
- rather than failing to import the batch, there should be an
error message indicating the group user had a validation error
fixes CNVS-4225
Change-Id: I2a20756d0464d833ce66e94a4d6c847bfb894f14
Reviewed-on: https://gerrit.instructure.com/18455
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
When unhandled ActiveRecord validation errors occur during SIS import
for a single record, it can prevent an entire batch from importing.
This change catches validation errors in Enrollment and
GroupMembership objects and re-raises them as ImportErrors so that
batch imports can continue.
Test Plan:
- create a batch import in which a user is enrolled in a class,
and is also in a group within that enrollment
- add something else to the batch (to test that the rest of the
batch is imported even when the error, below, occurs)
- import the batch
- mark the enrollment as deleted
- re-import the batch
- rather than failing to import the batch, there should be an
error message indicating the group user had a validation error
fixes CNVS-4226
Change-Id: I87078bd263f2b2487c5a8d41ff41c0ff71cbed95
Reviewed-on: https://gerrit.instructure.com/18271
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
closes CNVS-3417
test plan:
* after the migrations are run, ensure that every section has at
least one entry in CourseAccountAssociations in the database
* smoke test SIS imports
Change-Id: I261cad633788efbf4b0c64db34436ef695856fee
Reviewed-on: https://gerrit.instructure.com/17256
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
fixes CNVS-3199
test plan:
* have a section with a name ~250 characters long
* enroll a user in that section via sis
* it should not fail
Change-Id: I83a4537653a61ad0fd0bb9448a5f212fad90e35d
Reviewed-on: https://gerrit.instructure.com/17090
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Reviewed-by: Duane Johnson <duane@instructure.com>
fixes #CNVS-2172
found every place I could in the codebase
that passed some kind of input into a finder
that looked up a pseudonym with an sis_user_id.
Then wrote specs to pass in integers (which in
all cases turned up the error described
in the ticket), and
made sure to do explicit casts on the inputs
until the specs didn't break anymore.
TEST PLAN (unsure of how to trigger from UI):
1) Take a pseudonym that already exists and has
an sis_user_id that could be an integer (is
made up of all numeric characters).
2) In the console, try to create a new
pseudonym with the same sis_user_id, but use
an integer instead of a string.
3) you SHOULD NOT get a postgres error
telling you that you need to do an
explicit type cast
4) you SHOULD get a validation error telling
you that 2 pseudonyms cannot have the same
sis_user_id.
Change-Id: Ic69da6a40609a50234a5388af02dd1fbeb021214
Reviewed-on: https://gerrit.instructure.com/16925
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
fixes #CNVS-2516
test plan:
- create two custom roles based on the same type
- with a SIS import, try to enroll a user in a course with
both roles (i.e., two rows in enrollments.csv)
- make sure both enrollments are created
Change-Id: I09cef8a5dfbcac6e3d7fd5bd6dadbb1cebcab384
Reviewed-on: https://gerrit.instructure.com/16499
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Adam Phillipps <adam@instructure.com>
closes #CNVS-1078
test plan:
- create custom roles in an account
- sis import:
- put the custom role name in the 'role' column in a SIS
enrollment import, and ensure the role_name is assigned
in the Enrollment
- ensure only valid roles can be assigned this way
(must be defined in the course's account or parent
account, and must not be inactive)
- sis export:
- ensure custom role names are exported in the 'role' column
of the SIS enrollment export and provisioning reports
Change-Id: Ib8b4c129d451023fa51c73747baadd42cb305338
Reviewed-on: https://gerrit.instructure.com/15868
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Adam Phillipps <adam@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
refs #11188
test plan: sis import a user with an email. then do another sis import
that only changes the user's email. the change should show up
immediately in the ui and in sis csv export reports.
Change-Id: I14efdf052ea7d2688ef07fff6ce25f76c6deb034
Reviewed-on: https://gerrit.instructure.com/15479
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
it was doing a find_by_sis_source_id(nil)
also, don't find deleted terms
fixes#11465
test plan: create an enrollment term in the UI, separate from the
default term, with no sis id. do a course sis import and don't specify a
term, it should get assigned to the default term not the one you
created.
Change-Id: I4ff8e9df35b7f29237840314f118fe249d699838
Reviewed-on: https://gerrit.instructure.com/14738
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
if an SIS update tried to conclude an observer of a student
who was already concluded, that observer enrollment ended up
duplicated with one concluded enrollment and one active.
Test Plan:
* Create a student with an observer
* In an sis update conclude the enrollments with the student's row first
(you can look at the example in the added spec)
* Both users should be concluded and only have one enrollment
closes#11050
Change-Id: Iaf63f005397f7b93403e0fd032335bf0a1f364d2
Reviewed-on: https://gerrit.instructure.com/14094
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Adds support for optionally viewing documents with Crocodoc.
closes#9865
Test plan:
* configure the crocodoc plugin
* add an assignment that allows file uploads
* make a submission for that assignment with a pdf or doc or ppt
- on the 'submission details' page, opening a preview of the
assignment should display it in crocodoc
- speedgrader should display the submission in crocodoc too
* make a submission with odt or rtf
- the submission should be displayed with scribd or google docs
* if you disable the crocodoc plugin, submissions could continue being
previewed in google docs or scribd
Change-Id: I7dd2547f8e2d907c98ebe894a7f1ee9d58f1e030
Reviewed-on: https://gerrit.instructure.com/13668
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
test plan:
* create a SIS import that enrolls a student after an observer
* the student's enrollment should not have an associated user ID
Change-Id: I6b05e1c7cc4a634a77795b5d8ddc7ba269810479
Reviewed-on: https://gerrit.instructure.com/13776
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
do it all in a single update query - avoids extraneous touches as well
as several queries to load data that can be checked in the update
test plan:
* run grade publishing specs
* publish grades with async option
* import the results
Change-Id: I95a67cd1c4d7459cb0f28033421328da6de7113a
Reviewed-on: https://gerrit.instructure.com/12992
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
fixes#9768
test plan:
* create two accounts via SIS that reference each other (you'll need
to add the first account two times, once valid, the second changing
its parent to the other account)
* one of the accounts should cause a warning about a loop
Change-Id: Ieb1710a1d0b7d11c652d8e000dcf33b63d2b187f
Reviewed-on: https://gerrit.instructure.com/12749
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
test plan:
* configure parallel sis imports
* start a massive import
* kill the job
* in script/console, reset the batch's workflow_state to 'created',
and call process on it
* when it completes, the counts should be accurate, and not
doubly include what was processed the first time before it was
killed
Change-Id: Ib49a5b56fa9db88908ab386aa869614972ae66eb
Reviewed-on: https://gerrit.instructure.com/12420
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
test plan:
* enable parallel SIS imports in plugin settings
* do an SIS import that will parallelize
* it should not hang because the jobs don't run
Change-Id: Ib3c32c2ee2378946b7a79c8df2f404dd94eb957c
Reviewed-on: https://gerrit.instructure.com/12242
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
test plan:
* create two courses via SIS CSV
* enroll a user in course 1 via SIS CSV by course_id only
* xlist course 1's default section to course 2
* enroll another user in course 1 via SIS CSV by course_id only again
* the user should end up in the xlisted section in course 2
Change-Id: I3a114d70a109594a56915b705de77b87a177e966
Reviewed-on: https://gerrit.instructure.com/12122
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
test plan:
* import a csv file with all-caps headers
* it should still work
Change-Id: Ifeac18095e742e974f3709915d6d21f8c5d2d11a
Reviewed-on: https://gerrit.instructure.com/11206
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Cody Cutrer <cody@instructure.com>
test plan:
* import a course via SIS
* publish and conclude the course via the UI
* re-import the course via SIS (still 'active') status
* the course should stay concluded
Change-Id: Ieba993d1aac65a3e0872edec5deda2731f70e165
Reviewed-on: https://gerrit.instructure.com/10422
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
fixes#8310
test plan:
* load some sis data for terms/courses/sections/enrollments into an
account
* do a batch mode import of an enrollments.csv with just the headers,
no data
* all enrollments should be deleted, but courses and sections should
be left alone
Change-Id: I05beb577bfb4573e97f8b398aeb4bfff5df78a4f
Reviewed-on: https://gerrit.instructure.com/10265
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
fixes#8402
test plan:
* create a user via SIS with no e-mail
* add an e-mail address and remove it from the user's profile
* add the same e-mail address, and confirm it this time
* re-import the user via SIS, with the e-mail address this time,
but differing in case
* it should succeed, and the e-mail address should be update to
the new case
Change-Id: Id54f413bdf6b3c066530c53715141c7507d18f9b
Reviewed-on: https://gerrit.instructure.com/10552
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
test plan:
* upload a mostly valid sis csv with lots of rows, and one bad quote
way down
* you should get a (somewhat) helpful "Malformed CSV" error, rather
than an entirely unhelpful "Error while importing CSV. Please contact
support."
Change-Id: I58c87c9e9078db53b1e18ab988d841b89b091ea7
Reviewed-on: https://gerrit.instructure.com/9946
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
fixes#7159
test plan:
* import two users with the same e-mail into the same account
* the user should not get a merge notification e-mail
Change-Id: Ic9f3a3948df1e2a877fcd8e79f3d05d86a2d1e5b
Reviewed-on: https://gerrit.instructure.com/8774
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
The group SIS CSV was being detected by an optional setting
so this makes it check on only required headers.
It also adds counts to the "report" shown on the sis page.
Test Plan:
* Import groups without the account_id header
* It should be successful and show the count in the UI
closes#5730
Change-Id: I9b8cc27be82499fbb7b8778eacab183fb77a5126
Reviewed-on: https://gerrit.instructure.com/8828
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
fixes#7147
test plan:
* enable open registration
* invite a new user to a course
* that user should not show up in the account's user list
* finish registering the user
* the user should now show up in the account's user list
Change-Id: I60790c213671a7c16a52082602725a2468ad2dc4
Reviewed-on: https://gerrit.instructure.com/8502
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
fixes#7117
test plan:
* import a user as a student and a teacher in the same course
* import two students, and one observer that is associated with both students
Change-Id: Iab49f6884f84a2e947edb5d5380f8d31b23c5afc
Reviewed-on: https://gerrit.instructure.com/8427
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
test plan:
* import the same deleted enrollment several times; only one should be
in the db
Change-Id: If067578fc1b42e501e4c43243f9745654b633701
Reviewed-on: https://gerrit.instructure.com/8417
Reviewed-by: Zach Wily <zach@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
run the file completely through the parser before giving it to the
importers, and fail-fast with a useful error message
test plan:
* import an SIS file with mismatched quotes
* you should get a warning about malformed CSV, not a reference
to "Error while importing CSV. Please contact support."
Change-Id: If785910e76bce6fcc2efe5507febcbfc14727a05
Reviewed-on: https://gerrit.instructure.com/8095
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
test plan:
* create a user via the UI
* add another pseudonym
* delete the extra pseudonym
* import a new user via SIS import with the login of the pseudonym
you deleted
* it should create a separate user for the SIS user, instead of
resurrected the deleted pseudonym onto the original user
Change-Id: Ia1ff1490bae82fc4daef3f91b05293d7c79cc50b
Reviewed-on: https://gerrit.instructure.com/7975
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
test plan: run the migration
Change-Id: I75184edd64ddee384de485c443d4c7eee45cc26f
Reviewed-on: https://gerrit.instructure.com/7895
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
test plan:
* create a csv file with invalid UTF-8
* import it
* you should get an immediate error, without processing any records
Change-Id: Ife6c851d798613c161449dc8f6233dfadd67be4e
Reviewed-on: https://gerrit.instructure.com/7885
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
so that it's case insensitive
also fix some latent CommunicationChannel.find_or_initialize_by_path
discovered by specs for this
test plan:
* run the specs
* use forgot password with a username that differs in case
* self-register a new user multiple times (without confirming)
with e-mail addresses that differ in case
Change-Id: I476325f591c997fc8d50d5f38480177f732f07a7
Reviewed-on: https://gerrit.instructure.com/7724
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
test plan:
* import a user via SIS
* do another SIS import that lists the same user, and a new user
* the e-mail addresses for each user should be correct
Change-Id: I0a95804056e38bdfd36f24c0c78187d601940451
Reviewed-on: https://gerrit.instructure.com/7833
Reviewed-by: Zach Wily <zach@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
test plan:
* create a user via SIS import with an e-mail address
* add a different e-mail address to the user
* re-import the user via SIS, and set their e-mail address
to the second e-mail you added, but with a different case
* the user should have a single e-mail address, the one set
in the second SIS import
Change-Id: Iab6b16e0b37cfb8caac3faa453be570b99621a9e
Reviewed-on: https://gerrit.instructure.com/7784
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
most specs load the files in the right order anyway, but the
nightly plugins build was getting the wrong order and breaking.
this calls out the dependency correctly
testplan: n/a
Change-Id: Ie543492a3c02cb587e578cbc2b12badb663acd63
Reviewed-on: https://gerrit.instructure.com/6999
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Bryan Madsen <bryan@instructure.com>
* expose sortable name directly to the user
* don't downcase it
* use a LOWER(sortable_name) index for postgres
* set sortable name as "last_name, first_name" explicitly for SIS imports
* populate sortable name intelligently in the UI
Change-Id: I476641f4817e27a11b573d91f102c5a74d3eba26
Reviewed-on: https://gerrit.instructure.com/6512
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
Reviewed-by: Ryan Florence <ryanf@instructure.com>
fixes#5573, #5572, #5753
* communication channels are now only unique within a single user
* UserList changes
* Always resolve pseudonym#unique_ids
* Support looking up by SMS CCs
* Option to either require e-mails match an existing CC,
or e-mails that don't match a Pseudonym will always be
returned unattached (relying on better merging behavior
to not have a gazillion accounts created)
* Method to return users, creating new ones (*without* a
Pseudonym) if necessary. (can't create with a pseudonym,
since Pseudonym#unique_id is still unique, I can't have
multiple outstanding users with the same unique_id)
* EnrollmentsFromUserList is mostly gutted, now using UserList's
functionality directy.
* Use UserList for adding account admins, removing the now
unused Account#add_admin => User#find_by_email/User#assert_by_email
codepath
* Update UsersController#create to not worry about duplicate
communication channels
* Remove AccountsController#add_user, and just use
UsersController#create
* Change SIS::UserImporter to send out a merge opportunity
e-mail if a conflicting CC is found (but still create the CC)
* In /profile, don't worry about conflicting CCs (the CC confirmation
process will now allow merging)
* Remove CommunicationChannelsController#try_merge and #merge
* For the non-simple case of CoursesController#enrollment_invitation
redirect to /register (CommunicationsChannelController#confirm)
* Remove CoursesController#transfer_enrollment
* Move PseudonymsController#registration_confirmation to
CommunicationChannelsController#confirm (have to be able to
register an account without a Pseudonym yet)
* Fold the old direct confirm functionality in, if there are
no available merge opportunities
* Allow merging the new account with the currently logged in user
* Allow changing the Pseudonym#unique_id when registering a new
account (since there might be conflicts)
* Display a list of merge opportunities based on conflicting
communication channels
* Provide link(s) to log in as the other user,
redirecting back to the registration page after login is
complete (to complete the merge as the current user)
* Remove several assert_* methods that are no longer needed
* Update PseudonymSessionsController a bit to deal with the new
way of dealing with conflicting CCs (especially CCs from LDAP),
and to redirect back to the registration/confirmation page when
attempting to do a merge
* Expose the open_registration setting; use it to control if
inviting users to a course is able to create new users
Change-Id: If2f38818a71af656854d3bf8431ddbf5dcb84691
Reviewed-on: https://gerrit.instructure.com/6149
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
fastercsv is not supported in 1.9, instead csv in the stdlib has been
modified to be api compatible with fastercsv. in this first step, we
alias CSV to FasterCSV when running under 1.9. This allows 1.8.7 to
continue working with no changes.
Change-Id: I34c3a9031b6f4946380510e4833203e29a05073a
Reviewed-on: https://gerrit.instructure.com/5835
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
notable changes:
* nothing is processed as a sis-import blocking error now. bad imports now
result in warnings, while just skipping bad data
* we no longer check for duplicates before going to the database
Change-Id: Iedc96b29d92caccdc6a71ae1de8100a1c82dd137
Reviewed-on: https://gerrit.instructure.com/5724
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
It wasn't handling removing the association on the transition to deleted state.
Change-Id: Ia9f0714b8eba200531b5e2764c1c4f152f8ff8be
Reviewed-on: https://gerrit.instructure.com/5380
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Also clear old whitespace that was missed before this fix
closes#5448
Change-Id: I6096685223c43bfec9fd0c669c737af422687e10
Reviewed-on: https://gerrit.instructure.com/5354
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
update_all's update hash doesn't have any magic performed on bare Time
objects; it assumes any Time object it's given is already in UTC. using
a TimeWithZone object (regardless of timezone), which Fixnum#ago and
friends happen to return, is still fine.
Change-Id: I297b2a3211b896b5225ebcfaaee3c1eb56e55fb6
Reviewed-on: https://gerrit.instructure.com/5351
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
optimize crosslisting itself:
* use ids where possible to avoid unnecessarily loading up objects
* Don't worry about keeping track of if we need to save; we're gonna
save anyway
* update account associations on any account change, not just root
account change (if you re-crosslist a section from one sub-account
to another sub-account, the users may no longer be associated with
the first sub-account).
optimize sis imports:
* really batch up update_account_associations
Change-Id: Ic0fbe1601afcbcd3e6540e69febc2e6a1a94157f
Reviewed-on: https://gerrit.instructure.com/5137
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
* Attempt to cache the course/section between rows
* On section change, calculate the new account associations *once*,
and use the new User#update_account_associations incremental mode
for strictly new enrollments
* Keep the account_assocations account_chain cache intact between
calls
* Fall back to global update_account_associations if too few
enrollments in the section, if not a strictly new enrollment, or
if the user is already going to be globally updated
Change-Id: I884a394aef4f4b81f4472ee3a57f89c1f72ae371
Reviewed-on: https://gerrit.instructure.com/5136
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
if an enrollment belongs to a course/section pair, and then the section is
crosslisted to another course while the enrollment data stays with the
original course/section pair, the next time that enrollment gets imported
canvas shouldn't bail, but should instead pick the new crosslisted course
Change-Id: Ieb93ab19be6ee620fcf39463fc31db82048f3ea8
Reviewed-on: https://gerrit.instructure.com/5125
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
Previously we were allowing 'active' to override any term date restrictions.
Now we interpret 'active' to mean 'active excluding any other date-based
restrictions'.
Also fixed a bug that was causing inactive to be treated as active.
Change-Id: Idf435e7b5c129c410f828b6393d7d50fc366fbde
Reviewed-on: https://gerrit.instructure.com/5119
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
This was causing "full batch mode" to delete things that it shouldn't.
Fixes#5084
Change-Id: I6e19820ae9daf0b2096fd569451669ad2bb5f52f
Reviewed-on: https://gerrit.instructure.com/4820
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
This was already a small issue if the job queue was on a different
database driver than the main database, and it'll become more important
as more AR connections are introduced.
Change-Id: I204becadd32bb935df096e8c937a04bb6962f0b2
Reviewed-on: https://gerrit.instructure.com/4601
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
if a user crosslists via the ui, we don't want the SIS to blow away those
changes. fixes#4840, refs #4815
Change-Id: Ia3f844b3a33d9c9a6e9433dc79ce74e433f1f389
Reviewed-on: https://gerrit.instructure.com/4162
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
In particular, @batch.finish is necessary for the "blocked" job to finish.
Also, don't use send_later_if_production, just do it everywhere. By default
it won't get here anyway, so it's okay for tests.
Change-Id: I53bb4276769af481e3787073ddebb7ea5e5a5ed1
Reviewed-on: https://gerrit.instructure.com/3769
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
For "small" imports (<5000 users), this improves wall-clock time
only a bit, but is much much nicer to the db. The larger the
import, though, the more the improvement here.
Be careful that we only use transactions on things that wouldn't
block other imports running in parallel, or even progress updates
running on the same thread.
Also be smarter about how often we update progress, since that
transaction can have a considerable impact on wall clock time.
refs #3752
Change-Id: I8a79bc530f5433858ccf7d62d33b98ba25e98ffa
Reviewed-on: https://gerrit.instructure.com/3613
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
* Controlled by plugin settings; default 1
* Will re-balance incoming CSVs into at least <parallelism> new CSVs
(per import type)
* If there is a single CSV, it will be split optimally
* If there are multiple CSVs, no such guarantees are made
* Each CSV will be processed as a separate DelayedJob
* Accounts are not parallelized
* All other import types are parallelized within their own type. I.e.
all courses will be imported before any sections are started
Change-Id: Ifa6fe4e63812be95db999bdf963ab7cc1b15b75d
Reviewed-on: https://gerrit.instructure.com/3582
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
refs #4487
This consolidates our ErrorReport class with our ErrorLogging mechanism,
it's all in ErrorReport now and you call ErrorReport.log_error or
ErrorReport.log_exception to both create an ErrorReport object, and call
the hooks similar to what ErrorLogging did so that plugins for other
error handling mechanisms can be injected.
ErrorReport has a category field now, similar to how ErrorLogging used
to take a type. the /error_reports UI can filter by category.
The plugin interface was designed with Hoptoad integration in mind, but
it should be pretty general.
Change-Id: I59f7a0d44cf4b6215ad13ff92d30e1d1af607b74
Reviewed-on: https://gerrit.instructure.com/3577
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
it is not a suitable unique identifier, as evidenced by actual data
Change-Id: Ib2b2f6d523b2ad052b4eff9a35c10a6e62d394c5
Reviewed-on: https://gerrit.instructure.com/3565
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
refs #3752
also avoid an account lookup if possible
Change-Id: I8ea6b276cfdcfa86c5c70bd0051bca62737d7d77
Reviewed-on: https://gerrit.instructure.com/3564
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
* Only save the pseudonym once
* Avoid saving the user and pseudonym if nothing actually changed
* Related to that, since we skipped updating sis_batch_id
individually, we still need to do that, so do it en-masse instead.
updated_at will *not* get updated now.
These changes make re-importing the same data 3x faster than the
original import (it used to be less then 2x as fast as the original
import).
Change-Id: Ia6ea32cdeb2f4aebe041d8a51b9efe6c89bc819c
Reviewed-on: https://gerrit.instructure.com/3558
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
And avoid broadcast_policy callbacks for them
(save_without_broadcasting only affects the object you call it on,
and not everything that gets automatically saved along with it).
Change-Id: Iaeff7c038bfb2516449eaa12c8b87a1256374ca5
Reviewed-on: https://gerrit.instructure.com/3533
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
* Pre-set objects we already have that get used later
* Compare ids instead of objects to avoid loading stuff we don't
have (and don't need)
Change-Id: Idf7b701a5ae294c973b3c058dfa4d4f4d0b752bc
Reviewed-on: https://gerrit.instructure.com/3527
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Whitmer <brian@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Rob Orton