Commit Graph

524 Commits

Author SHA1 Message Date
Cody Cutrer e8a9fc7b13 rubocop: Rails/ReadWriteAttribute
[skip-stages=Flakey]

mostly done manually, since many of these calls weren't even necessary,
or can easily be replaced with `super`

Change-Id: I1a5a38a1332a26a11c6041a005d691730206fc10
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/344440
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Cody Cutrer <cody@instructure.com>
2024-09-23 18:16:48 +00:00
Jeremy Stanley 58641d62c1 SIS import: don't set pronouns if account setting is off
test plan:
 - render and view API docs and note that the SIS CSV
   format document should indicate the "pronouns" column
   will be ignored if the "Enable Personal Pronouns"
   account setting is off
 - do a SIS import of a file with that column present
   when the setting is off and ensure it does nothing

flag=none
closes FOO-4680

Change-Id: I003238321d2770cfdb2e55214db1d615dae190fe
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/355814
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2024-08-23 17:25:16 +00:00
Jason Perry 9a29bc546b Don't check cached batch when checking stop
Closes FOO-4633
flag=none

Test Plan:
  * Start a large batch mode SIS import
  * Abort it (via API or console) part-way through
  * Observe that parallel importers are aborted rather
    than continuing to run

Change-Id: I48fda5274d061b4f30661a5cb247bffd54e3a532
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/355098
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Jason Perry <jason.perry@instructure.com>
QA-Review: Jason Perry <jason.perry@instructure.com>
Product-Review: Jason Perry <jason.perry@instructure.com>
2024-08-15 16:33:10 +00:00
Kai Bjorkman 2f3429aa65 set grading_standard when inherited on conclusion
flag=default_account_grading_scheme
closes EVAL-4375

Test Plan:
    - create two account grading schemes with different grade letters
    (So you can easily see the difference)
    - Set one as the account default
    - Create a course and do not set the grading schemes
    - Create an assignment and set display grade as letter grade
    and set the grading scheme as the course default which will be the
    account default
    - grade a few students on that assignment and ensure a letter
    grade is shown for the final grade and assignment grade
    - conclude the course
    - change the default account grading scheme to the other scheme
    - ensure all of the original letter grades and final grades are
    still the same
    - bonus test concluding the course with the course update endpoint
    (the ui uses the destroy endpoint)
    https://canvas.instructure.com/doc/api/courses.html#method.courses.batch_update

Change-Id: I87f4589d9457830021da249b8d1853ab32742927
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/352866
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Sleyder Zuleta <sleyder.zuleta@instructure.com>
Reviewed-by: Derek Williams <derek.williams@instructure.com>
QA-Review: Sleyder Zuleta <sleyder.zuleta@instructure.com>
Product-Review: Ravi Koll <ravi.koll@instructure.com>
2024-07-22 15:41:53 +00:00
Maya Tyner ee15c491ab create temp enrollment pairing in SIS import
temp enrollments through SIS now associate with a pairing

fixes FOO-4542
flag=none

test-plan:
- pass new rspec tests
- compare temp enrollment creation in SIS to UI

Change-Id: I65cd5ad1efcaf67a7d22aa52ca2002b11a688490
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/350687
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Maya Tyner <maya.tyner@instructure.com>
Product-Review: Maya Tyner <maya.tyner@instructure.com>
2024-06-26 16:54:41 +00:00
Cody Cutrer 5e6e8b3091 rubocop: Rails/FindBy
[skip-stages=Flakey]

Change-Id: Idb7623681def2b06abab65c80c5411d3dde99cd5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/349801
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2024-06-11 19:46:18 +00:00
Cody Cutrer 7c3b10e9e3 bundle update rubocop
[skip-stages=Flakey]

mostly Style/MapIntoArray (and then additional autocorrects enabled
after that was done). besides dynamo_query.rb and localized cop
disabling, everything was autocorrects

Change-Id: I9d663a6921d778bba38db2d179732964b4a26437
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/344905
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Build-Review: Cody Cutrer <cody@instructure.com>
2024-04-10 16:31:10 +00:00
Jeremy Stanley e12c32e548 make term override dates sis sticky
test plan:
 - create a terms.csv that sets override dates for various
   enrollment types for a term
 - import it
 - manually change a date for one enrollment type
 - re-import the csv
 - the manual change should stick
 - re-import the csv with the "Override UI Changes" box checked
 - the value should revert to the one from the csv

flag=none
closes FOO-4144

Change-Id: I6b3a1a0377f6aa78f6029ba310c6367be1115090
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/339596
Reviewed-by: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Migration-Review: Isaac Moore <isaac.moore@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2024-02-02 20:35:04 +00:00
Jeremy Stanley 1055e582c8 don't unnecessarily copy files before uploading
Rack::Test::UploadedFile, in addition to looking like something
that isn't intended to be used in production code, creates a copy
of the file that is passed in. This slows down things like
reports and exports, and isn't needed for our use cases,
which all involve synchronous uploads (and do not involve #append).

a note about unlinking: in a few places, the code calls #unlink
on the Rack::Test::UploadedFile instance, which (1) did not
delete the file passed into that thing, only the copy it made;
and (2) was unnecessary since that class deleted its copy itself

also note that Rack::Test::UploadedFile is still being used
to simulate uploaded files in tests... as is intended.
but no longer using it just to wrap a `content_type` around
a file we are uploading from the backend

test plan:
 - smoke test the following:
  - content export
  - account report
  - sis import
  - submission export

flag=none
closes FOO-4215

Change-Id: Ife83373be5920958eb7bdf14c2da1a56708ca11e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/338855
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2024-02-02 19:32:30 +00:00
Jeremy Stanley 7e9cf002f0 reduce priority of template copies in sis imports
the same way we already did for blueprint syncs.
customers don't want these spun-off jobs to starve
out the originals

note that I have already created the
'sis_imports_course_templates_num_strands" setting
in production with a default value of 8

test plan:
 - enable template courses
 - set up a template in an account
 - run a sis import that creates a course
 - the template course copy job should be created at
   priority 25 (below the sis import's priority of 20)
 - the migration should run in the
   "sis_imports_course_templates" n-strand

flag=none
closes FOO-4198

Change-Id: Icecdbb4ca201d5f861b1c2befa4fd58a10ab913e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/339363
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2024-02-02 19:11:50 +00:00
Jacob Burroughs 3b7130c161 Remove a lot of settings
[ignore-stage-results=Flakey Spec Catcher]

refs AE-551

Change-Id: If7b5191c20cfadc438cdc2bc8b489eb2806582fe
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/334831
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
2024-01-09 21:32:17 +00:00
Cody Cutrer c32d30f662 bundle update rubocop
[skip-stages=Flakey]
[skip-crystalball]

and apply corrections from new/updated cops:

 * Layout
 * Lint/AmbiguousRange
 * Lint/RedundantSafeNavigation
 * Style/HashEachMethods
 * Style/HashSyntax
 * Style/InverseMethods
 * Style/RedundantLineContinuation
 * Style/RedundantParentheses
 * Style/RedundantReturn
 * Style/SuperWithArgsParentheses
 * Style/SymbolProc

Change-Id: I787e46f5b1c89e0fedebd5452ba74567d66954c1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/334291
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Build-Review: Aaron Ogata <aogata@instructure.com>
Migration-Review: Aaron Ogata <aogata@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2023-12-06 14:25:02 +00:00
Jeremy Stanley 4b951832f9 fix "deleted sis login" error message
if you attempt to enroll a user with a deleted pseudonym
via sis import, the error message currently says
"Attempted enrolling with deleted sis login X", where X
is the Canvas ID of the pseudonym. That number doesn't mean
anything to SIS users. If we're mentioning a "login", we
should use what SIS imports call "login_id" there
(iow, the `pseudonyms.unique_id` column)

test plan:
 Prerequisites: A user with a SIS id enrolled in a course
  with a SIS id.
 - Delete the user with a SIS import (so really just
   deleting the login from the user).
 - Make an enrollments SIS import attempting to enroll the user
   in the course with the SIS id from the deleted login. Check
   the error attachment for the SIS import. it should use the
   sis "login_id" column instead of the canvas ID.

flag=none
closes FOO-4018

Change-Id: I58f120a2e433cc6f495af16fdbc8e50ab700b92c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/333276
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-11-20 17:53:10 +00:00
Jeremy Stanley 16acb2ac30 fix group CSV imports w/o course_id or other context
test plan:
 - have group category containing a group, both in course context
 - run a SIS import modifying the group, but don't provide
   course_id, account_id, or group_category_id in the CSV file,
   for example, use only group_id,name,status
 - the group and group category should both stay in the course

flag=none
fixes FOO-3839

Change-Id: I0b90c1f97c717b0286d3870950ae08e07050016d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/329334
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Hulse <michael.hulse@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-10-04 19:30:26 +00:00
August Thornton 63bd7405a5 add additional warning states to sis enrollment importer
fixes FOO-3890
flag = none

Test plan:
Prerequisite: A course and user with a SIS ID
- Navigate to the user page for the user, and select to
  Delete from ALL accounts.
- Run a SIS Import file to enroll the user into the existing course,
  making sure the limit_section_privileges column is included
- Notice that the SIS Import doesn't show any enrollments added,
  but there is a warning included

Change-Id: Icbeba5a5262b439676cfe629e7f1569498b8bfeb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/328752
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
2023-09-27 16:52:50 +00:00
Jeremy Stanley a1b16a163c clear AccountCacher in change_sis_id imports
test plan:
 - have redis enabled
 - have a course in an account with a SIS ID
 - do /api/v1/courses/X/enrollments and see that this ID
   is shown in `sis_account_id`
 - change the sis id of the account via a change_sis_id.csv
   import
 - repeat the API call and the new SIS ID should be seen

flag=none
fixes FOO-3728

Change-Id: Idf5177f7710247b10c2a7231494129f7281e843e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/328597
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-09-25 19:09:57 +00:00
August Thornton 1966a6eec0 Add ability for SIS Import to support temporary enrollments
closes FOO-3326
flag=temporary_enrollments

test plan:
- create an enrollments sis csv file with a temporary enrollment
- import the file and process the data (ensure jobs are running)
- verify that the enrollment is created with the correct
  temporary_enrollment value

Change-Id: I8fbbd4860394f5a96989fb5d0136175e4aa47795
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/327516
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Hulse <michael.hulse@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
2023-09-12 23:51:09 +00:00
Jeremy Stanley 4029ac103d fix failure when diffed sis batch column types change
if a diffed csv import changes the column definitions, our
intended behavior is to skip diffing for that type. this was
broken in FOO-3574 but only because the variable name changed :(
for the warning text and there was no test coverage of that code path

test plan:
 - column header changes should bypass diffing and run a full
   import of the type(s) that were changed
 - the entire import should not fail

flag=none
fixes FOO-3749

Change-Id: I6ed9a27cf63d6ff1176e549002f650b19d472386
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/325798
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jason Perry <jason.perry@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-08-23 15:52:03 +00:00
Jeremy Stanley 244ba09355 add diffing_user_remove_status option
this feature was requested by multiple customers
at InstructureCon 23 hack night

test plan
 - import a SIS user batch using a diffing_data_set_identifier
 - remove a user from the csv file and re-import
   using the same diffing_data_set_identifier
   while supplying the "diffing_user_remove_status=suspended"
   option
 - the user should be suspended rather than deleted

flag=none
closes FOO-3710

Change-Id: I5a0acf0677c0512a334060f917a3fa4fdf32fcd4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/323910
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ajmal Esa <ajmal.esa@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-08-03 19:04:15 +00:00
Cameron Ray 8e25fecec6 Rename DueDateCacher to SubmissionLifecycleManager
closes EVAL-3285

[fsc-timeout=30]

We will need to follow this up next release with a commit
to remove the alias `DueDateCacher = SubmissionLifecycleManager`.

flag=none

Test plan
- Tests pass

Change-Id: Ib3566e9d581f5826ab07ee2265277bb75ba02ee0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/320685
Migration-Review: Isaac Moore <isaac.moore@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Christopher Soto <christopher.soto@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cameron Ray <cameron.ray@instructure.com>
Product-Review: Cameron Ray <cameron.ray@instructure.com>
2023-07-11 14:43:12 +00:00
Michael Hulse fc74d431a7 fix group_category to match context_id of imported group
fix bug with “Student Groups” group_category and sis-imported
groups: when importing groups, groups will belong to the same
course as the group_category they are assigned to

refs FOO-3573
flag=none

test plan:
- create sub-account course with SIS ID
- create groups.csv with SIS ID
- perform SIS import
- verify visibility in UI and API
- reset course
- verify removal of groups
- perform SIS import again
- verify visibility in UI and API
- for more details see FOO-3573

Change-Id: Ic0b90bcafabeface46e782384e9f7ba52734ff32
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/321657
Reviewed-by: Michael Hulse <michael.hulse@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Hulse <michael.hulse@instructure.com>
Product-Review: Michael Hulse <michael.hulse@instructure.com>
2023-07-10 21:44:12 +00:00
Jeremy Stanley 4f0daaae90 enhance diff generator to support multiple files
currently CSV diffing mode doesn't operate unless there is only
one file of each type in the previous and current packages.
some schools have separate files for e.g. student and teacher
enrollments. this PS allows them to use diffing mode by
matching up the files to diff based on filename.

test plan:
 - import a SIS package containing student_enrollments.csv
   and teacher_enrollments.csv, specifying a
   diffing_data_set_identifier
 - make some changes to both files
 - re-import the package, using the same
   diffing_data_set_identifier
 - ensure the correct results happen. note that you can get
   the diffed output from the SisBatch via
   sis_batch.downloadable_attachments(:diffed)
   and verify we did the expected thing
 - if all filenames of a given type do not match between
   the previous and current imports, no diffing will be
   performed (the new files will be imported in their entirety)

flag=none
closes FOO-3574

Change-Id: If81b05c6b945bbed40eb2de95d52faf51dd2ac60
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/320623
Reviewed-by: Michael Hulse <michael.hulse@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-06-26 17:28:41 +00:00
Maya Tyner 33ad8ae3f7 stop SIS courses from associating with deleted accounts
courses now set to root account if the old account is deleted

fixes FOO-2199
flag=none

test plan:
1. run sis imports on deleted courses from deleted accounts

Change-Id: I671467856025ed3a5f78f0206d4047e93979f483
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/319797
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
2023-06-09 16:51:44 +00:00
Jeremy Stanley b5d6e73823 add job_ids to sis_batches and parallel_importers
so when things go wrong--but not so wrong there is an ErrorReport--
we can find the logs for the job(s). (typically there will just
be one job per SisBatch or ParallelImporter, but when failed
jobs are retried, it may be useful to see the logs for each
attempt.)

test plan:
 - run a sis import job
 - in the Rails console, look at the SisBatch and the linked
   ParallelImporters
 - they should have populated `job_ids` which correlates with
   what was written in the logs

flag=none
closes FOO-3478

Change-Id: I1e684e23fcbd73cea63bc20c5bc591b03143a42a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/319264
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Migration-Review: Isaac Moore <isaac.moore@instructure.com>
Migration-Review: Aaron Ogata <aogata@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
2023-06-07 17:09:46 +00:00
Jacob Burroughs 7dcc507d0a Rubocop for ruby 3.1
[skip-stages=Flakey]

Change-Id: I6abefdfa9fed6dd4525c8786e93efa548b3710f2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/319603
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Build-Review: Jacob Burroughs <jburroughs@instructure.com>
Migration-Review: Jacob Burroughs <jburroughs@instructure.com>
2023-06-06 16:44:26 +00:00
Cody Cutrer 502aa068da rubocop: Layout/Multiline*LineBreaks
[skip-stages=Flakey]

Change-Id: I24912809267088bc19a4082c49bc5f78ca7b76bb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/317302
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Build-Review: Jacob Burroughs <jburroughs@instructure.com>
Migration-Review: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2023-05-01 19:31:42 +00:00
Cody Cutrer dbfff50c2d rubocop: Layout
addresses minor changes in cops in rubocop 1.49

[skip-stages=Flakey]

Change-Id: I79503c905f59752d67391e70909e73441323faee
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/315160
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2023-04-12 14:57:56 +00:00
Cody Cutrer b2b0bff902 rubocop: Style/TernaryParentheses, Style/RedundantParentheses
[skip-stages=Flakey]

Change-Id: I8d860eedd8b199ed3adf0e2bf9162b43096c5347
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/315130
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Build-Review: Cody Cutrer <cody@instructure.com>
2023-04-12 14:56:48 +00:00
Jacob Burroughs b8940e7594 Fix some more bulk updates
refs AE-158

Change-Id: I3d1e4c35d85ccf978de0b09d5700575e255e979a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/315450
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
2023-04-11 15:07:24 +00:00
AJ Esa b5837b0821 prevent restoring nested sub_account within deleted sub_account
Prevent the restoration of a nested sub_account within a sub_account
when imported via SIS import.

fixes FOO-3400
flag = none

test plan:
- Create sub_account and nested sub_account via SIS import
- Delete both accounts via SIS import
- Restore only the nested sub_account via SIS import
- Should get a warning on SIS import UI that nested sub_account
  cannot be restored because parent account has been deleted

Change-Id: I62682c3a30b971930f3a3abe830ddb695582766b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/314099
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jason Perry <jason.perry@instructure.com>
QA-Review: Ajmal Esa <ajmal.esa@instructure.com>
Product-Review: Ajmal Esa <ajmal.esa@instructure.com>
2023-03-30 15:57:56 +00:00
Jeremy Stanley dad3f86eef update sis_batch_id on unchanged students' linked observers
test plan:
 1. have a student linked to an observer via UserObservationLink
 2. enroll the student in a course via a batch mode SIS import
    that imports the student alone
    - ensure the observer is enrolled as expected
 3. run another sis import that explicitly creates the
    observer enrollment and the student enrollment
 4. re-run the import CSV from step 2
    - ensure the observer's enrollment is still active

flag=none
fixes FOO-3424
fixes gh-2172

Change-Id: I47db58382a94805d48c6beb9ea5ab2ba4b6c3fcc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/314467
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Jason Perry <jason.perry@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2023-03-29 15:54:47 +00:00
AJ Esa be376216db Enrollment completed time set to when enrollment is completed
When an enrollment is made active, then completed, made active again
and at least one minute later completed one more time (all via SIS
imports), the completed at time will show the original completed
time instead of the second completed at time. This commit makes it
so that the completed at time is always set to the time when the SIS
import is done.

fixes FOO-3316
flag=none

test plan:
 - Make sure delayed jobs are running
 - Import an enrollment via SIS import and make account status active
 - Update CSV to change student status to
   deleted_last_completed
 - Take note of completed date and time on student membership page
 - Wait at least a minute
 - Run SIS import CSV, setting the student status to active
 - Run SIS import CSV, setting student status back to
   deleted_last_completed
 - Time noted from earlier should differ from new completed time on
   student membership page

Change-Id: I46d26dad7ff479798710866b9cc8d682ede8217e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/309778
QA-Review: Ajmal Esa <ajmal.esa@instructure.com>
Product-Review: Ajmal Esa <ajmal.esa@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
2023-02-06 16:49:21 +00:00
Jacob Burroughs 618a6e702e Kill require_dependency
refs AE-31

flag=none

test plan:
- verify Canvas boots in CD
- verify no influx of new errors in CD

[fsc-timeout=1]
[ignore-stage-results=Flakey Spec Catcher]

Change-Id: I26663e32d9452c198c87b957b026170ee3e4bc93
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/305073
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Isaac Moore <isaac.moore@instructure.com>
Product-Review: Isaac Moore <isaac.moore@instructure.com>
2023-01-09 19:51:01 +00:00
Isaac Moore d6584b490a Remove unnecessary require statements
closes AE-30

flag=none

test plan:
- verify Canvas boots in CD
- verify no influx of new errors in CD

[fsc-timeout=30]

Change-Id: Ifa04bebe1b09f01c6d3b8b2d8f3bb424759730f5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/308067
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Isaac Moore <isaac.moore@instructure.com>
Product-Review: Isaac Moore <isaac.moore@instructure.com>
Build-Review: James Butters <jbutters@instructure.com>
2023-01-04 21:38:21 +00:00
Jeremy Stanley 4eafc60275 enable integrity check on SIS import downloads
and remove the retry on MalformedCSVError since that's weak
compared to SHA512 and has proven not to be robust enough to
detect repeated or elided data in the file

with `integrity_check: true`, `Attachment#open` will redownload
the file (once) if the SHA512 hash doesn't match, so we're
not losing anything by ripping this code out

flag=none
closes FOO-3256

Change-Id: Iea899f4995b5d093406e7748c9768ed65a167a02
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/307128
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2022-12-15 18:20:22 +00:00
August Thornton d4d70dbc06 invalidate adminable accounts user cache for admins sis import
fixes FOO-3099
flag = none

test plan:
 • have a target user / subaccount with associated SIS ids
 • import an admins.csv with user / role / account info
 • verify user details for that user includes enrollments for
   that new account
 • verify admin badge is present in global nav that includes
   that new account user enrollment for the role chosen to
   be imported (will be a named link to the account) after
   clicking on the admin tab

Change-Id: Id590197ffd48a4ac08398bd5388657a64b5d735c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/302956
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
2022-10-12 14:31:27 +00:00
Ievgenii Matkivskyi 579a256060 Fix sets imported user's unique_id
Closes SOS-2816

Test plan:
 - Import canvas user with canvas_password_notification flag on
 - Check imported user unique_id

flag=none

Change-Id: I337fecee7242269b5cb640e80deb08a7b6cc9ee6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/300867
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Product-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
Reviewed-by: Tamas Nagy <tamas.nagy@instructure.com>
2022-09-29 15:34:36 +00:00
Ben Rinaca 49f20e9e0b expand sis hook method
refs FOO-3028

flag = none

an extension to this hook method needs the pseudonym to make its
decisions

test plan:
- nothing is different in canvas-lms

Change-Id: I6ce95d60eb6f7b52842e0a7b6bfd6dad750b6906
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/297321
Reviewed-by: Ajmal Esa <ajmal.esa@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Jesse Poulos <jpoulos@instructure.com>
QA-Review: Ben Rinaca <brinaca@instructure.com>
2022-08-31 20:24:12 +00:00
Ievgenii Matkivskyi f966d077c8 Revert "Add converter to utf-8 for imports"
flag=none
Closes SOS-2811

This reverts commit b95dafe9b0

Test plan:
  - Run Roster import job from sistemic

Reason for revert: Invalid byte sequence happening in production.

Change-Id: I5af469957f955aea393c7f8393648c0c9be477fe
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/296804
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Tamas Nagy <tamas.nagy@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
QA-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Product-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
2022-07-25 11:09:28 +00:00
Ben Rinaca ab179e16a4 catch import errors on user preparation
refs FOO-2771

some of the hook implementations may raise errors and we want that to
only fail the single row rather than the whole batch

Change-Id: I7d7ad5f730f8254abe1ad18b326685820206d7e4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/295166
Reviewed-by: Sean Scally <sean.scally@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ben Rinaca <brinaca@instructure.com>
Product-Review: Ben Rinaca <brinaca@instructure.com>
2022-07-07 23:10:48 +00:00
Ben Rinaca 058ca6167e hook user creation in sis import
refs FOO-2771

allow for plugins to override user creation behavior when importing
users in SIS.

the column `home_account` in users.csv can contain a boolean value that
a plugin can use to signal if a new user should be created instead of
using a) the existing user for an existing login or b) an existing user
found for a new login via the `other_user` hook method.

test plan:
* open source canvas should have no change, regardless of the value in
  the `home_account` column

Change-Id: I973906371ca8d97ce1b6bbf972b1f90375113375
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/293109
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ben Rinaca <brinaca@instructure.com>
Product-Review: Ben Rinaca <brinaca@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
2022-06-22 20:55:48 +00:00
Ievgenii Matkivskyi b95dafe9b0 Add converter to utf-8 for imports
flag=none
Closes SOS-2428

Test plan:
 - Upload archive with Users.csv in ISO encoding
 - Upload archive with Users.csv in ASCII encoding

Change-Id: I680c0b343c3b0374fd7e207eb1f43e063b1c104a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/287235
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Product-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
Reviewed-by: Tamas Nagy <tamas.nagy@instructure.com>
2022-06-07 09:54:21 +00:00
Luis Oliveira c061bcd02e Add course paces republish on SIS import
closes LS-3111
flag=course_paces

test plan:
- Enable course pacing for a given course
- Update that course's start date via SIS import
- Check that the course's paces have been republished

Change-Id: Ibc3e14d6a249a151f5e584d95e18ed8bc62f027b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/290973
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Robin Kuss <rkuss@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Luis Oliveira <luis.oliveira@instructure.com>
2022-05-03 15:01:37 +00:00
Spencer Olson deb4370dca limit simultaneous sis import due date recache job runs
closes EVAL-2376
flag=none

As part of this change, the number of simultaneously running enrollment
importer due date recache jobs per shard is capped at 10. In
order to change this, run:

```
Setting.set(
  "DueDateCacher#recompute_for_sis_import_num_strands",
  "20"
)
```

Test Plan:
1. create an assignment
2. Verify in a rails console that submissions were created by
   DueDateCacher for the assignment:

   Submission.where(assignment_id: <id>).count
   => should be greater than 0

3. Run a SIS import that adds a new enrollment to a course (for a user
   that has not yet been added to the course), and verify submissions
   are created for any assignment that is assigned to the newly-added
   enrollment:

   Submission.where(
    course_id: <id>,
    user_id: <enrollment's user_id>
   ).count
   => should be greater than 0

Change-Id: I5f28b5bfdc7f419bc2fc450aa4cc23fe2c28a51f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/288944
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Shafovaloff <ashafovaloff@instructure.com>
Reviewed-by: Kai Bjorkman <kbjorkman@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
2022-05-03 13:45:51 +00:00
Ben Rinaca ee39cb20c0 abort sis import when attachment download fails
there apparently is a case where the file is fine on app server disk,
but somewhere in round-tripping inst-fs the file becomes empty, and then
sis import sees no objects in the attachment and thinks it should delete
all of the existing objects during the import.

i couldn't find a codepath where the file could become empty, but we
have at least two reported cases where that has happened. so let's go
ahead and prevent the bad thing from happening. and if this results in
us then being notified of other occurrences then we will have more cases
to investigate a root cause for.

fixes FOO-2506

Change-Id: I2b7ae9ca11e690c4bfe3337d2e7d82a6d03dd2d7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/288865
QA-Review: Ben Rinaca <brinaca@instructure.com>
Product-Review: Ben Rinaca <brinaca@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
2022-04-04 22:11:24 +00:00
Jackson Howe fe9c4d7eb1 Don't (sis) import observers into blueprint courses
fixes LS-3073
flag = none

Test plan:
 - Create users.csv, courses.csv, and enrollments.csv files to create
   a user, course, and enroll the user into the course as an observer
   (or get them from me)
 - Zip the files and upload them at accounts/id/sis_import
 - Expect to get this warning - "Observer enrollment for abc not
   allowed in blueprint course def"
 - Visit the new course's users page and expect the user to not have
   been enrolled

Change-Id: I51bfefcac73219df21ff0c12a23e3e0a717d451f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/288739
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Jackson Howe <jackson.howe@instructure.com>
2022-04-04 15:48:33 +00:00
Vusal Ismayilov 20c21bbe8c Add notification field for users CSV
Closes SOS-1939

Test plan:
 - Import canvas user with canvas_password_notification flag on
 - Click the link on mail
 - setup password for the user
 - login with the credentials

flag=none

Change-Id: If0e3522cdb95b0f53e3b21ae9fc9c99a68cf0ab6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/285599
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Vusal Ismayilov <vusal.ismayilov@instructure.com>
Product-Review: Vusal Ismayilov <vusal.ismayilov@instructure.com>
2022-03-11 08:26:21 +00:00
Stephen Kacsmark d75f15e840 Revert "Add converter to utf-8 for imports"
flag=none
refs SOS-2711

This reverts commit 3b246102c2.

Reason for revert: SIS UTF-8 errors happening in production.

Change-Id: Ifea49be6e3a8e62c90d65d1cbc67cf6a700de485
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/285821
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
QA-Review: Balazs Komaromi <balazs.komaromi@instructure.com>
Product-Review: Stephen Kacsmark <skacsmark@instructure.com>
2022-03-04 20:13:55 +00:00
Ievgenii Matkivskyi 3b246102c2 Add converter to utf-8 for imports
flag=none
Closes SOS-2428

Test plan:
 - Upload archive with Users.csv in ISO encoding

Change-Id: Id827b71649cf8e402fbdf87d1735eb54b90ee1bd
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284959
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
Reviewed-by: Kozma Jozsef <jkozma@instructure.com>
QA-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Product-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
2022-02-16 14:40:48 +00:00
Ievgenii Matkivskyi f250fff25f Add possibility to import course with friendly_name
flag=none
Closes SOS-2627

Test plan:
 - Upload archive with courses.csv for elementary account
 - courses.csv should contain field with friendly_name
 - course name after importing should have value as friendly_name

Change-Id: I7ab7b1e146fb1b5838ea00a4e67de3fb2c7e52b1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283743
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Richard Zana <rzana@instructure.com>
Reviewed-by: Balazs Komaromi <balazs.komaromi@instructure.com>
QA-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
Product-Review: Ievgenii Matkivskiy <ievgenii.matkivskiy@instructure.com>
2022-02-09 14:02:33 +00:00