* Tag all GraphService metrics (and failed batch add/remove metrics)
with new tag "sync_type" which is either "partial" or "full". Partial
sync is only one step -- step_partial_sync -- so we put the code to
change the tag to "partial" there.
* Partial syncs that turn into full syncs will have the metrics under
"sync_type: full", but there are additional counters
"partial_into_full" and "partial_into_full_throttled" to denote that
this has happened.
closes INTEROP-6798
flag=microsoft_group_enrollments_syncing
Test plan:
- monkey-patch statsd by adding this to some file, e.g. syncer_steps.rb
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD increment: #{args.to_json}"
end
def count(*args)
puts "DEBUG STATSD count: #{args.to_json}"
end
end
- add an enrollment and remove another to a course that has microsoft
sync set up and has already been synced once. either watch the job
logs if you are running jobs (you have to wait the debounce delay in
this case), or, more easily, open a console and run:
microsoft_sync_group.syncer_job.run_synchronously(:partial)
- look at the logs and check that all "DEBUG STATSD" lines for
microsoft_sync.* metrics include a tag "sync_type: partial" (except
those that contain a microsoft_sync_step, as those convey the same
info)
- add and remove users in the Microsoft admin console. Then kick off a
full sync with:
microsoft_sync_group.syncer_job.run_synchronously(:partial)
Again check that all "DEBUG STATSD" lines except for the exceptions
above include tag "sync_type: full"
- clear the ms_group_id on a group. Kick off a partial sync with
microsoft_sync_group.syncer_job.run_synchronously(:partial)
check the output. Make sure the "partial_into_full" metric has been
incremented and that all other "DEBUG STATSD" metrics (with the
exceptions noted above) have "sync_type: full"
Change-Id: Ie36f7b24f6d81c52ae750db78cb354b8aa9d5dc9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268132
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Tucker Mcknight <tmcknight@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
closes DEMO-189
flag=none
test plan:
- create an assigment with a submission type other than
online_quiz
- send a student submission for the assignment
- as a teacher/admin, you should be able to update the
submission type and the submission type options
from the UI and the REST API
(PUT /api/v1/courses/:course_id/assignments/:id)
- create an assignment with submission type "online_quiz"
- as a teacher/admin, you should be able to update the
submission_types field via the REST API
- as a student, take the online quiz
- as a teacher/admin, you should not be able to update
the submission_types field via the REST API
Change-Id: I97853407569894de03974dc3a29ecea91b22f50d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266015
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Susan Sorensen <susan.sorensen@instructure.com>
Reviewed-by: Stephen Kacsmark <skacsmark@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
fixes INTEROP-6894
flag=none
Test plan
- Have an LTI 1.x tool that allows grade passback
(Google or Office work. Office is easier to build.)
- Set up an assignment to link to the tool
- Have MRA set up and have a user from another shard
in your course
- Send a request to https://<canvas>/api/lti/v1/tools/:tool_id
/grade_passback with the below XML using the following
commands (from Canvas or another tool that has Oauth)
consumer = OAuth::Consumer.new(:tool.lti_key, :tool.lti_secret)
client = OAuth::AccessToken.new(consumer)
client.post(url, xml, 'Content-Type' => 'application/xml')
- The XML should look like below, but make sure to put in a url
and a sourcedId for the assignment/cross-shard user combo
- Get the sourcedid from lis_result_sourcedid
in the Form Data from the network request of the LTI
tool launch
- Verify the request gets a 200 instead of a 422
<?xml version = "1.0" encoding = "UTF-8"?>
<imsx_POXEnvelopeRequest xmlns="http://www.imsglobal.org/services/ltiv1p1/xsd/imsoms_v1p0">
<imsx_POXHeader>
<imsx_POXRequestHeaderInfo>
<imsx_version>V1.0</imsx_version>
<imsx_messageIdentifier>999999123</imsx_messageIdentifier>
</imsx_POXRequestHeaderInfo>
</imsx_POXHeader>
<imsx_POXBody>
<replaceResultRequest>
<resultRecord>
<sourcedGUID>
<sourcedId>(real sourcedid)</sourcedId>
</sourcedGUID>
<result>
<resultData>
<downloadUrl>(url)</downloadUrl>
<documentName>(name)</documentName>
</resultData>
</result>
</resultRecord>
</replaceResultRequest>
</imsx_POXBody>
</imsx_POXEnvelopeRequest>
Change-Id: I7dc4c45f9fd77d5d02509df32a3166769e3444e3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268311
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Tucker Mcknight <tmcknight@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
To properly test calls to InstStatsd::Statsd.* you should add the
following to your docker-compose.override.yml under "environment":
INST_STATSD_HOST: localhost
INST_DOG_TAGS: '{}'
As of this commit, I have done that and then changed all instances of
Statsd.count() in the lib/microsoft_sync directory to "increment" (buggy
old version) one-by-one, and run the corresponding spec files (also
graph_service_spec.rb for program code graph_service_http.rb). Each of
them then failed with the "wrong number of arguments (given 3, expected
1..2)" error.
refs INTEROP-6798
flag=microsoft_group_enrollments_syncing
Test plan:
- kick off a sync that will make two or more users deletions
(e.g. removing a teacher may count as 2 -- deleted as owner and as
member)
- open a console and monkey-patch statsd:
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD increment: #{args.to_json}"
end
def count(*args)
puts "DEBUG STATSD count: #{args.to_json}"
end
end
- Watch that read quota and write quota are incremented by the correct
amount (number of users removed), with the msft_endpoint tag equal to
"batch_group_remove_users"
Change-Id: I00cdcb7ccfbff5cf157aa3249f1c234adade7e1d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267810
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
refs INTEROP-6798
flag=microsoft_group_enrollments_syncing
Test plan:
- specs should be enough
Change-Id: Ie848d0844a4966c3ed6c14e313e10f102eb4b7c8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267808
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
New statsd metrics that see how exactly how much read and write quota we
are using, at allow us to split it up by endpoint. I will do quota
metrics for batch requests in a later commit. Note that teams endpoints
have their own throttling limits, they do not use these quota points.
See https://docs.microsoft.com/en-us/graph/throttling
refs INTEROP-6798
flag=microsoft_group_enrollments_syncing
Test plan:
- open a console and monkey-patch statsd:
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD increment: #{args.to_json}"
end
def count(*args)
puts "DEBUG STATSD count: #{args.to_json}"
end
end
- run `UserMapping.delete_all` so we can see some requests that look up
AADs
- Create a new course (or use an existing course with no group on the
Microsoft side) with >= 1 teacher and >= 2 students
- Run a synchronous sync for new course. The following requests should
cause the following quota usages (read,write):
GET education/classes (1,0)
POST education/classes (1,1)
PATCH groups/{id} -- first time is to set course data fields (1,1)
GET /users (1,0) -- endpoint normally (2,0) but we use select
GET groups/{id}/members (2,0) -- normally (3,0) but we use select
GET groups/{id}/owners (1,0)
PATCH groups/{id} -- this time to add users. Write quota=1. Read
quota is variable: number of owners and members added, divided by
3 (rounded up.) Teachers are owners and members so count as two.
e.g., for a course where we are adding 1 teacher and 2 students,
read_quota = (2*teachers + students)/3
= (2*1 + 2)/3
= (4/3)
= rounded up to 2
- Add or remove students such that the write quota for adding users will
be different. For instance, make the course have 1 teacher and 1 student
(write quota should be just 1). Delete the course on the microsoft side
and rerun a sync and check that the PATCH request to add users now has
that correct write quota.
Change-Id: Ic69b02d643e1d25f356be559974bf19455dac5da
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267723
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
refs INTEROP-6798
flag=microsoft_group_enrollments_syncing
I ran a sync and it failed due to a 404 in
update_group_with_course_data -- the group hadn't been created after
(5+15+100 = 120) seconds, or 2 minutes. Increase that to 6 minutes.
Test plan:
- only changing numbers, specs should be enough, you can run
a sync and make sure it still works if you like.
Change-Id: Ia45714503c0d293a7f680cee32489685b5d3835d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267818
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
fixes INTEROP-6763
flag=none
Test plan
- Have an LTI 1.x tool that allows grade passback
(Google or Office work. Office is easier to build.)
- Set up an assignment to link to the tool
- Turn off your delayed jobs
- Send a request to https://<canvas>/api/lti/v1/tools/:tool_id
/grade_passback with the below XML using the following
commands (from Canvas or another tool that has Oauth)
consumer = OAuth::Consumer.new(:tool.lti_key, :tool.lti_secret)
client = OAuth::AccessToken.new(consumer)
client.post(url, xml, 'Content-Type' => 'application/xml')
- The XML should look like below, but make sure to put in a url
and a sourcedId from a real assignment/user in your instance
- Make sure to note the time you sent the request
- Restart your delayed jobs after a few minutes. Ensure the
submission has the original request time as the submitted_at
and not your Canvas-delayed timestamp
- Get the sourcedid from lis_result_sourcedid
in the Form Data from the network request of the LTI
tool launch
- Any URL should theoretically work...
- Ensure you can also send the submitted_at date and
verify it shows up correctly
<?xml version = "1.0" encoding = "UTF-8"?>
<imsx_POXEnvelopeRequest xmlns="http://www.imsglobal.org/services/ltiv1p1/xsd/imsoms_v1p0">
<imsx_POXHeader>
<imsx_POXRequestHeaderInfo>
<imsx_version>V1.0</imsx_version>
<imsx_messageIdentifier>999999123</imsx_messageIdentifier>
</imsx_POXRequestHeaderInfo>
</imsx_POXHeader>
<imsx_POXBody>
<replaceResultRequest>
<resultRecord>
<sourcedGUID>
<sourcedId>(real sourcedid)</sourcedId>
</sourcedGUID>
<result>
<resultData>
<downloadUrl>(url)</downloadUrl>
<documentName>(name)</documentName>
</resultData>
</result>
</resultRecord>
</replaceResultRequest>
</imsx_POXBody>
</imsx_POXEnvelopeRequest>
Change-Id: I9dded1dfff4afd39428b085bcb1298f1f949dd4c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266236
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
fixes LS-2325
flag=none
test plan:
- Create a page
- In the content of the page create a link to the course home by clicking
the dropdown next to the link button in the RCE menu, then clicking
Course Links, expand Course Navigation, then select Home
- With the Home link in the content, save the page
- Export the course
- Import the course content into a new course
- Verify the link in the imported page content is linked to the new page's
home navigation and not the old course
- Also verify just using the Copy Course from Import Course Content also
changes the home navigiation link as expected
Change-Id: Ib7779a4c42f9669ac4da6408a2fbd453ae31a039
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267975
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jackson Howe <jackson.howe@instructure.com>
QA-Review: Jackson Howe <jackson.howe@instructure.com>
Product-Review: Eric Saupe <eric.saupe@instructure.com>
You will need to allow CORS (via browser extension) and allow
Mixed content, which occurs if the initial request is secure over
HTTPS (required by Notorious plugin), but HTTPS and HTTP content
is loaded to display the web page (required by canvasrceapi)
See: Enable the Notorious Plugin (Replacement for Kaltura)
in Instructure wiki
fixes FOO-2095
refs FOO-1659
flag = none
test plan:
• Within your root account:
• Account.default.settings[:enable_offline_web_export] = true
• Within Course Details for your test course:
• Verify "Allow course content to be downloaded and viewed offline"
• Locally you will need to have Canvas RCE API set up"
• https://github.com/instructure/canvas-rce-api
• If you're running docker, follow doc/docker/developing_with_docker.md
• Canvas RCE API section
• Create a course and add a wiki page to it
• In the wiki page use the RCE toolbar to Upload/Record Media, this
will require the use of the Notorious plugin /plugins/kaltura
• Record and Save, leaving a title or the generated placeholder text
• Ensure to save and publish the wiki page
• Add the wiki page to a published module
• Go to the course settings page and in the navigation tab
hide the Files tab and |Save|
• Go to the modules page and click export course content in
the top right
• Download the zip file and open the files folder to verify you see
only the media files that were embedded into the body of the page
that was linked via the module
Change-Id: I42c8631be489a7ced3f50e88210d1b67a200ea03
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267745
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
If a pseudonym is found during a user import but the SIS ID doesn't match,
this flag would override the old (or non-existent) non-matching SIS ID
with the new SIS ID in the upload.
closes SOS-1922
Test plan:
- Do a SIS import
- Change the SIS ID of one user in the input file
- Run SIS import again
- Check the SIS ID of the changed user in Canvas
flag = none
Change-Id: I2b29904c1e34e7a7beebd5f41176e35f3fb54ef1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267749
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
GraphService now contains only endpoints and endpoint-specific code.
GraphServiceHttp contains code common to all/multiple Microsoft
endpoints, such as stats, logging, pagination, batching.
In the next part I move some code around within the two files and make
the functions in GraphServiceHttp private if they should be.
To review use git show --color-moved
refs INTEROP-6798
flag=microsoft_group_enrollments_syncing
Test plan:
- see part 3
Change-Id: Iaa17cf067a8c7287c0fed2d253186362c04e173f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267788
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Fix sort of a race condition that happens when the User changes tenant
or login_attribute while the job is running. In this case, the job may
look up the AADs with the wrong tenant or login_attribute. This prevents
us from writing those incorrect values to the DB.
closes INTEROP-6721
flag=microsoft_group_enrollments_syncing
Test plan:
- add a 'byebug' right before 'result_hash' in
GraphServiceHelpers.users_upns_to_aads
- clear out UserMappings (MicrosoftSync::UserMapping.delete_all) and run
a sync
- when the breakpoint hits, in a console modify the Account settings'
microsoft_sync_tenant. Make sure to save the changes to the database.
- continue sync
- sync should fail with AccountSettingsChanged. check that the
UserMapping table is still empty.
- Go to the course settings integration tab and you should see that error.
- remove the byebug and add a byebug in SyncerSteps#ensure_user_mappings
after 'users_and_upns = users_upns_finder.call'
- repeat steps above for microsoft_sync_login_attribute instead of
tenant
- remove byebugs, clear out UserMappings, and run a sync just to make
sure it still works
Change-Id: I4fad4722279bb44bf6ddd7e608deac3607dc97a2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267004
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Siteadmin is not a place for normal canvas things. If a course is
created on siteadmin it generally causes problems or leads to confusion
when knowing which things apply to it. This adds a validation to just
fail if the root_account is siteadmi
test plan
- on siteadmin try to create a course
- it should fail
flag=none
fixes VICE-1616
Change-Id: Ifd4de5403f7962cc2d79a30a4d5febd4b358c309
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267340
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
calculations were changed to use only bigdecimal
instead of multiplications and divitions by float
fixes EVAL-1744
flag=none
test plan:
- create a course with at least one student enrolled.
- create an assignment set to display grade as percentage
with 100 points posible.
- load speedgrader and enter one of the following
values (57, 55, 50).
- navigate to the assignment submission API
creating an GET request using this URL
assignment/api/v1/courses/:course_id/assignments/
:assignment_id/submissions.
- note that score and given_score display an well rounded
decimal value.
Change-Id: I968b683694f4ab18d8bc5a09d0be9013af589b6a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267354
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Dustin Cowles <dustin.cowles@instructure.com>
Reviewed-by: Adrian Packel <apackel@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Jody Sailor
reduce dependence on mutation of rows and adding of
offsets to calculate assignment indices. filter out
non assignment header columns earlier via regex.
closes EVAL-1747
flag=none
test plan:
- have a course with multiple assignments and assignment groups
- export the gradebook to csv
- swap some of the assignments so they have a non-default
ordering in the csv
- move one of the assignment group columns so that it is between
normal assignments
- alter/add grades for the assignments that were moved
- import the modified csv via the gradebook import
- check that all the assignments with modified grades were
picked up and that there is no new assignment for the
assignment group that was placed between assignments
Change-Id: I416763e3806d636ef86740e1608c26d26c29f8ee
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267038
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Adrian Packel <apackel@instructure.com>
Reviewed-by: Dustin Cowles <dustin.cowles@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Jody Sailor
no more step_initial and initial_step
changes to state_machine_job_spec are nothing more than
s/step_first/step_initial/ (plus getting rid of initial_step method)
closes INTEROP-6581
flag=microsoft_group_enrollments_syncing
Test plan:
- run a sync to make sure it still works
Change-Id: Id582b7caacd3204af9ae0f72664d2fc98d51dbb3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267185
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
also have to fix several cases where cross-shard queries have indeterminate order
Change-Id: I1faf5953787859c24481e51dad3cf60d6d332a96
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266738
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-1761
flag = granular_permissions_manage_courses
test plan:
- builds pass
- migration runs successfully
Change-Id: I421d3794d0b0de62c60043237ba54f7aa24f7127
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267318
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
- If throttled in partial sync, switch to full sync. Trying partial sync
again will likely only make throttling much worse.
- Remove stale TODOs
- Clean up/DRY up SyncerSteps tests for retrying intermittent errors
- Add two specs to microsoft_sync/group_spec.rb
closes INTEROP-6581
flag=microsoft_group_enrollments_syncing
Test plan:
- To reduce your wait time, set delay to kick off partial sync job to
something small like 6 seconds (default is 10 minutes):
Setting.set('microsoft_group_enrollments_partial_syncing_debounce_minutes', 0.1)
- Modify SyncerSteps#step_partial_sync to add the following:
err = StandardError.new
err.define_singleton_method(:retry_after_seconds) { 12 }
err.send(:extend, MicrosoftSync::Errors::Throttled)
raise err
- Add/remove an enrollment which will kick off a partial sync
- Start jobs and watch output. It should turn into a full sync job
(step_full_sync_prerequisites) after a delay of 12 seconds.
Change-Id: Ib1d0379e6b333e20cba326566e13da9efd4bc783
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266696
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Optimization to not do a full sync every time an enrollment changes, but
instead only add/remove enrollments for the users that were
added/removed.
refs INTEROP-6581
flag=microsoft_group_enrollments_syncing
Test plan:
- Note: to view job logs below, I recommend using the
jobs_output_filter.rb script in the interop-team-scripts repo, e.g.
docker logs -f $(docker ps|grep canvas.*jobs|awk '{print $NF}') | \
jobs_output_filter.rb Microsoft immediate timestamp
- Setting.set('microsoft_group_enrollments_partial_syncing_debounce_minutes', 0.5)
- have a course with sync enabled but with ms_group_id empty (e.g., make
a new course and enable sync, or reset ms_group_id on an existing MicrosoftSync::Group)
- make an enrollment change (note: all emails must be confirmed for
enrollment changes to take effect, if using email as the UPN type setting). e.g.,
CommunicationChannel.where(workflow_state: 'unconfirmed').update_all(workflow_state: 'active')
- check Delayed::Job.last for the enqueuer job. It should be on strand
"MicrosoftSync::Group:10000000000001:enqueue_future_partial_sync" (etc.)
- check MicrosoftSync::PartialSyncChange table to see if there is an
entry for the enrollment
- have job container running and wait (0.5 minutes if you set the delay
above) until the job runs. Looking at the logs you should be able to
tell it turns into a full sync job (it will go run through all the full
sync steps)
- make another enrollment change
- check Delayed::Job again to see the job
- make another enrollment change before the last job runs
- see that there is still only one job, it has been debounced (moved up
in time)
- wait again and let the job run
- look at logs and see what changes it uncovers
- make sure those changes are correct and it has implemented them
- try different enrollment scenarios, including adding and removing
user multiple times before debounced job runs, and users who are (or
were) both teachers and students. check logs to make sure changes make
sense and group state on Microsoft side ends up correct -- with MS
admin console, or on a canvas console running
graph_service.list_group_members(group.ms_group_id)
and ...list_group_owners(...)
- make some enrollment changes and before the job runs kick off a manual
full sync
- the full sync should run and delete all the PartialSyncChanges because
it took care of them, so when the partial sync job runs, it should show
no changes (you should be able to tell from the logs that the partial
job runs and attempts no actions)
Change-Id: I8efa286b6cec56fda656e9dcb465f020ed711847
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265856
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
This change makes graphql endpoints functional even if
the FF is off
closes OUT-4348
flag=improved_outcomes_management
Test Plan:
- Setup an account with at least 1 outcome and
1 group
- Make sure the query returns same values
with the flag on and off
{
legacyNode(type: Account, _id: 1) {
... on Account {
rootOutcomeGroup {
outcomesCount
childGroupsCount
childGroups(first: 1) {
nodes {
_id
title
}
}
outcomes(first: 1) {
nodes {
... on LearningOutcome {
_id
title
}
}
}
}
}
}
}
- Repeat the process for an course instead of account
}
Change-Id: Ic0ef4dc6d42c4df7801ff7c3904c5d6a42049ac3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267150
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Chrystal Langston <chrystal.langston@instructure.com>
QA-Review: Martin Yosifov <martin.yosifov@instructure.com>
Product-Review: Augusto Callejas <acallejas@instructure.com>
if there is a relevant grading period, the title is always
included in aggregate column headers now. this feature flag
is no longer needed
closes EVAL-1739
flag=gradebook_csv_headers_include_grading_period
test plan:
- have a course with some grading periods
- select a grading period from the drop down
filter in the gradebook
- export to csv
- view the csv and ensure that the aggregate
column titles are appended with the grading
period title in parentheses
Change-Id: I8e5c0851ff1bd17915d4cf4423bd2338608f3e22
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266857
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Kai Bjorkman <kbjorkman@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
set users' gradebook column order preferences
as the sort order for assignment columns in the
gradebook csv export
closes EVAL-321
flag=gradebook_csv_export_order_matches_gradebook_grid
test plan:
- enable the feature flag
- in the gradebook, swap the order of some assignments
and then export to csv
- the assignment order should match the order you
created in the gradebook
- change some grades in the csv just exported
- import the modified csv and ensure the updated grades
are picked up
- now delete an assignment and repeat the export process.
said assignment should no longer appear in the export
- back in canvas, create a new assignment (do not make
any changes to the gradebook column order at this point)
- export once more, the new assignment just created should
come after all the original assignments (since it has
no preference)
- in all the above steps, the assignment group and course
total columns should come after all assignments
- test the 3 scenarios above again with the feature flag
disabled. the assignments should be ordered by
assignment group position and then assignment position
instead of the gradebook column order
- other things to consider: grading periods, override
scoring, custom columns, etc. exporter and importer
should behave as defined above with respect to the
feature flag status under all these circumstances
Change-Id: Ie248a7ffe663d28ddd1f2984e59a0f307ad41c52
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266000
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Adrian Packel <apackel@instructure.com>
QA-Review: Aaron Shafovaloff <ashafovaloff@instructure.com>
Product-Review: Jody Sailor
closes FOO-2054
flag=none
TEST PLAN:
1) enable AUA writing in a multi-region environment
2) each region should be successfully writing to it's
own regionally-suffixed namespace
Change-Id: Ia8d25c4f29c5b9081890081395c62b59a8a4a2c3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266977
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
We are going to have two kinds of jobs, full sync and partial sync. This
commit ensures that if a job starts while another one is in the process
of retrying, it will enqueue itself for later. It's only necessary to
queue one if one is not already enqueued though.
Multiple jobs of the same type can still be queued if one is actively
running for a period of time; I don't expect that to be common though.
StateMachineJob ended up being a little more complicated than I'd hoped.
(Now I know enqueuing jobs while another is being retried actually holds
of running those new jobs until the first has completed... I might have
been able to rely on this behavior and now have my own state checking
and such... but this is here now and is the last change I expect to make
to SMJ. I still sort of like how the job has the current state in this
way.)
closes INTEROP-6744
flag=microsoft_group_enrollments_syncing
Test plan:
- create a new group or have a group with no group on the Microsoft side
so running a sync will create a new group on the microsoft side
- change the last line of step_ensure_max_enrollments_in_a_course to add
debugging and and a delay:
puts "DEBUG_MEM_DATA: state=#{_mem_data}"
StateMachineJob::DelayedNextStep.new(:step_ensure_class_group_exists,
2.minutes)
Adjust the 2.minutes as necessary to efficiently do these steps.
- with a job container not running, run
group.syncer_job.run_later
group.syncer_job.run_later(:def)
group.syncer_job.run_later(:def)
group.syncer_job.run_later(:ghi)
group.syncer_job.run_later
- run a jobs container and check the logs. the initial_mem_state=nil
job (I'll henceforth referred to this as "the <nil> job") should start
and then wait on the DelayedNextStep. The first "def" job should run
and then drop itself. The second "def" job should run and enqueue
itself for later. The "ghi" job and <nil> job should run and enqueue
themselves for later, too.
('grep StateMachineJob' and looking for 'Processing',
'Completed', and 'INSERT', may be helpful). If reading
the logs is too difficult, you can just look at the jobs as
below.
- Get the jobs
def thejobs
Delayed::Job.where('strand like ?', '%StateMachineJob%').
order(:id).map{|j| [j.id, j.run_at, j.payload_object.args]}
end
Check that the fresh (step=nil) <nil>, "def" and
"ghi" jobs are enqueued for 1 second after the <nil> retried job.
- As the original <nil> job continues running and is retried (to go to
step_update_group_with_course_data), check that the fresh <nil>, "def"
and "ghi" jobs run and are immediately reenqueued for
after the retried (step != nil) <nil> job. `thejobs` should show the
the retring job (with first argument
:step_update_group_with_course_data) as well as the (def, ghi, nil)
jobs enqueued for one second after the retrying job.
- Enqueue another job
group.syncer_job.run_later(:ghi)
This won't actually run until all scheduled jobs
have run, but you should be able to see the new job with a run_at in
the past (before the others)
- Undo the debugging/delay added in step 2, delete all jobs:
Delayed::Job.where('strand like ?', '%StateMachineJob%').delete_all)
- Enqueue jobs with varying first state parameter as in the third step.
Look at the logs and see that they all eventually run to completion.
- Try to run a synchronous job (g.syncer_job.run_synchronously) while
another job is in the retrying state. It should raise an error if
there is no job with the same interation
- TODO: jobs enqueued when another is waiting with no duplicate job will
run in LIFO order. consider the consequences.
closes INTEROP-6744
flag=microsoft_group_enrollments_syncing
Change-Id: Ia01b0108f95b033361acb2381c830330b39a2849
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265474
QA-Review: Sean Scally <sean.scally@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
closes FOO-1836
closes FOO-1840
closes FOO-1841
refs FOO-1868
flag=none
TEST PLAN:
1) enable both pulsar reading and writing
for asset user access
2) do a bunch of accesses
3) let the compaction job run
4) the AUA records are properly aggregated
and the metadata for iterator state
is updated for both the postgres path
AND the pulsar path
Change-Id: I91fc1acbd85da76947f7c08746e96908de6fd35f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264161
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
so they can be queried efficiently
test plan:
- before checking this out, have a homeroom course set up,
as well as a course set to sync enrollments from a homeroom
- get this patchset and run migrations
- the existing homeroom course should still be a homeroom course,
and the course set to sync enrollments should also still do that
- you should be able to set up a new course as homeroom and
set a course to sync enrollments from it
closes LS-2256
Change-Id: Ic6dfdf4e0aa2907b788b9da62bbbad398a82d01f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266313
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Eric Saupe <eric.saupe@instructure.com>
QA-Review: Eric Saupe <eric.saupe@instructure.com>
because something broke in the CI system :(
This reverts commit 982515a123.
This reverts commit 74da1a5fa9.
Change-Id: Ie82085d5f6eaf654c42b78a4fb85507468b1ae16
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266281
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
so they can be queried efficiently
test plan:
- before checking this out, have a homeroom course set up,
as well as a course set to sync enrollments from a homeroom
- get this patchset and run migrations
- the existing homeroom course should still be a homeroom course,
and the course set to sync enrollments should also still do that
- you should be able to set up a new course as homeroom and
set a course to sync enrollments from it
closes LS-2256
Change-Id: I75d5adf8fb07017e49db363effdde91a930cb4b0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266014
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jeff Largent <jeff.largent@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Eric Saupe <eric.saupe@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
refs FOO-2029
flag=none
TEST PLAN:
1) run a console in a production environment
with user env var set
2) canvas can infer your user from the database
if one exists with the correct pseudonym
Change-Id: I7f4a36853d9b5307e0930cc36e6de292f0d98709
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266229
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
fixes INTEROP-6795
flag=none
Test plan
- Set up an LTI passback assignment
(office is what I used, but I'm
guessing there are simpler tools)
- Make sure you have a student in
the course
- Set the course conclusion dates
or term conclusion dates to be
in the past
- Set the section dates to be in
the present time and make sure
you select "the student can only
participate during these dates"
- Have the student submit the LTI
assignment
Change-Id: If237d23c41610c0f626b2cd1704c658931dfb18e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265954
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
This is part of the FIPS compliance project
Test plan:
- spec spec/lib/due_date_cacher_spec.rb passes
Change-Id: I8c2a3cc4cfa8828f06d325c8007796bf3a8bb1df
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266073
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
flag=annotated_document_submissions
closes EVAL-1639
Test Plan:
- Have a course with a teacher and a few students
- create an anonymous peer review student annotation assignment
- as one of the students annotate and submit the assignment
- as the teacher assign the other students to peer review the previous
submitted assignment (those students also have to submit before peer
reviewing is available)
- as those students ensure the student's name they are reviewing is
not attached to any of the annotations. Instead it will read 'Student'
- complete the peer review as those students and ensure the teacher can
see all the reviewers feedback and submitters annotations with names
attached
- Grade the student and ensure the student can see all the reviewers
comments and teachers feedback
Change-Id: Ic92a1ed4dcfe4d5420dbe53503fce8b34494aaae
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264953
Reviewed-by: Adrian Packel <apackel@instructure.com>
Reviewed-by: Edwin Ramirez <edwin.ramirez@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Shafovaloff <ashafovaloff@instructure.com>
Product-Review: Jody Sailor
fixes FOO-1898
[ignore-stage-results=Flakey Spec Catcher]
this commit touches known flakey specs,
but does not make them any flakier
if they have open registration off, we need to trust the SIS
if open registration is on, we ignore unconfirmed anyway, and
just create a new temp user that will get merged together
test plan:
* have open registration *off*
* add two users with confirmed email addresses (via SIS, etc.)
* log in as one of the users, and go to your profile
* add the other user's e-mail address (but don't confirm it,
since in reality you couldn't)
* log in as a teacher or an admin
* add a student to a course with the duplicated email
* it should only find the confirmed user
* add the user as an admin
* same result
Change-Id: I0a64c45876d7f49e3662f55496e76c5c37306376
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265171
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Change-Id: I078c5cbb3bc214df3409ff3b8661200e404709d1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265686
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
As long as the content being linked to from within the module
is published and active and the module is published and active
we allow media files to be generated in the offline export
content feature even if the linked to files are disabled
on the course navigation. Prior to this we were including images
and links, however media files are needed all the same.
fixes FOO-1659
refs OFFW-203
flag = none
test plan:
• Within your root account:
• Account.default.settings[:enable_offline_web_export] = true
• Within Course Details for your test course:
• Verify "Allow course content to be downloaded and viewed offline"
• Locally you will need to have Canvas RCE API set up"
• https://github.com/instructure/canvas-rce-api
• If you're running docker, follow doc/docker/developing_with_docker.md
• Canvas RCE API section
• Create a course and add a wiki page to it
• In the wiki page use the RCE toolbar to embed a video from your
computer on the page
• Add the wiki page to a module
• Publish the page and module
• Go to the course settings page and in the navigation tab
hide the Files tab and |Save|
• Go to the modules page and click export course content in
the top right
• Download the zip file and open the files folder to verify you see
only the media files that were embedded into the body of the page
that was linked via the module
Change-Id: I8f34dedc64c4b02aa0695ffe4f5ac797109ebab7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265487
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
closes INTEROP-6816
flag=microsoft_group_enrollments_syncing
Test plan:
- somewhere in graph_service.rb that will be run, for instance in
list_education_classes, raise an error:
raise BatchRequestThrottled.new('DEBUGTESTERROR',
['headers' => {'retry-after' => 10.days}])
- Run the sync and see form the logs that it is retried with a delay of
21300 seconds (6 hours minus 5 minutes). Look for logs like:
MicrosoftSync::StateMachineJob:MicrosoftSync::Group:10000000000128: Clipped delay 864000.0 to 21300.0
MicrosoftSync::StateMachineJob:MicrosoftSync::Group:10000000000128: handle_retry step_ensure_class_group_exists -> step_ensure_class_group_exists - 21300.0
- Change that retry-after to 5 hours
- Run the sync again (if you group is in a weird state, such as if you
Ctrl-C'd a synchronous run, it may be easiest to just delete the group
and recreate it) and check that it is retrying at 18000 seconds (5
hours)
- Change that retry-after to a negative number, re-run, and check that
delay is 0 (as a result it will retry immediately and fail
- Remove the 'raise' statement and run a sync and make sure from the
logs that the delays in DelayedNextStep (if used) are OK and the sync
completes successfully.
Change-Id: Ia0c7361a30551e3442c32a75acfe8e957e87d1e3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265425
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Currently if a job of any type is started when another job is retying,
it will be dropped; this will be fixed soon in another commit.
Test plan:
- in SyncerSteps#step_ensure_max_enrollments_in_a_course, add a line:
puts "DEBUG SMJ INITIAL_MEM_STATE: #{_mem_data.inspect}"
- in StateMachineJob#run, at the end, by the comment
"else: Trying to run a new job", add
puts "DEBUG SMJ: dropping #{initial_mem_state.inspect}"
- Make a new course or have a course which does not have a group on the
Microsoft side, so that when you run a sync job, it will need to
create the team (which requires waiting with DelayedNextStep)
- Enqueue two sync jobs with
group.syncer_job.run_later(:foo)
group.syncer_job.run_later(:bar)
- Start a jobs container and let the jobs run and look through the logs.
- When the first one pauses (with DelayedNextStep), you should see the
second run and then get dropped ("DEBUG SMJ: dropping :bar"). This
will be fixed in the next ticket I work on
- The first should job should end up running to completion.
- Run again with `group.syncer_job.run_later` (no args) to make sure the
sync still works with no initial_mem_state
refs INTEROP-6804
flag=microsoft_group_enrollments_syncing
Change-Id: I48b2bf0a89f33049889b28a663e3ce6c2ce038c7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265405
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
refs INTEROP-6804
flag=microsoft_group_enrollments_syncing
This is necessary because StateMachineJob will allow an initial
argument; the enqueue_future_sync strand would need tohave the argument
in it. This isn't necessary because we only use it in for enrollment
changes in the Group, so it's easiest to remove it from SMJ. Also SMJ is
big enough as-is.
Test plan:
- add/remove an enrollment on a course with a MicrosoftSync::Group
- in a console do this:
def thejobs(group)
Delayed::Job.where(
strand: "MicrosoftSync::Group:#{group.global_id}:enqueue_future_sync"
).to_a
end
- check thejobs(group) and see that there is one job with a run_at time of 10
minutes.
- Make another enrollment change. Check thejobs(group) again to see that
there is still only one job, but it has been moved up 10 minutes in the
future.
- Run a jobs container and wait 10 minutes (or run
Setting.set('microsoft_group_enrollments_syncing_debounce_minutes', 1)
and wait 1 minute) and make sure the sync happened (e.g. look at the
Microsoft admin to see that the users have been added/removed)
Change-Id: Ife02ab22c19fab54a2d96392523ef1e2ffc7499c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265388
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Also use the deregistration method in
a few specs in the hopes of reducing
spec flakiness.
Closes INTEROP-6813
Flat=none
Test Plan
- variable_expander_spec.rb passes
Change-Id: If4b7f53fcba7825b281fb7dc51fc5eaf4a45cb8e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265379
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
closes FOO-1897
test plan:
- create a new user, they should not be able to create an eportfolio
- add them to a course, now they can
- OR add them as an account admin, now they can
Change-Id: Ib18d1ccbfef7c5aff3e977c017ec471312867049
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263432
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
closes INTEROP-6797
flag=microsoft_group_enrollments_syncing
Test plan:
- have a course and group and sync it to make sure a normal sync is
still working.
- In a console, get a list of any 10 users that we will use in a request:
group = MicrosoftSync::Group.last
gs = group.syncer_job.steps_object.graph_service
users = gs.list_users.map{|x| x['id']}[0...10]
- Get ready to slam the MS API
def batch_of_bulk_adds(group, users)
reqs = (1..20).each_with_index.map do |i|
{
id: i.to_s,
url: '/groups/' + group.ms_group_id,
method: 'PATCH',
headers: { 'Content-Type' => 'application/json' },
body: {
"members@odata.bind" => users.map{|o|
"https://graph.microsoft.com/v1.0/directoryObjects/#{o}"},
"owners@odata.bind" => users.map{|o|
"https://graph.microsoft.com/v1.0/directoryObjects/#{o}"},
}
}
end
gs = group.syncer_job.steps_object.graph_service
gs.send(:run_batch,'post_teams',reqs) {false}
rescue => e
e
end
- Slam the MS API
results = 70.times.map{batch_of_bulk_adds(group, users)}
begin
gs.add_users_to_group(
group.ms_group_id, members: users, owners: users)
rescue => $non_batch_result; end
- Check out the results. The error messages should show we started
getting 429s after a while, and when 429s happened, the error should
have been BatchRequestThrottled instead of BatchRequestFailed:
results.map{|res| res.message[0..120]}
- Check that the BatchRequestThrottled errors included a
retry_delay_seconds value (I seem to be getting 150 for all):
results.map{|res| res.retry_after_seconds rescue 'n/a'}
- Observe the $non_batch_result and make sure it is a
HTTPTooManyRequests error and has a retry_delay_seconds.
- Make another course, or add a user to a course that is not in the
users you are using above. Open a second rails console. Run the
'70.times.map...' bit from above, and when it's got about half-way (35
requests), in the other console, kick off a sync that will cause some
users to be added (easiest to use group.syncer_job.run_synchronously).
Watch the logs and make sure there the job retries based on a retry from
a throttled error.
Change-Id: I4a4a81e979d2b7dabdfaf03cfcc32a9d163bfe37
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265214
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
refs INTEROP-6797
flag=microsoft_group_enrollments_syncing
Test plan:
- Test plan is in the last commit of this group of three for this
ticket. Commits are separated for ease of review.
Change-Id: Iae3e744b113c3f2e71f257533e8df5b3d3c8e3af
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265134
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Closes INTEROP-6801
flag=none
Test Plan:
- Run the `doc:api` rake task and make sure it succeeds
- Visit /doc/api/file.tools_variable_substitutions.html
and verify the last four expansions no longer
have a documented default parameter. Also verify
these expansions note they are only functional if
a query param is present in the request.
Change-Id: I71ddc1cd20dfa0fd120a8747db41d11fed1fc1a2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265267
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Minimizes calls (and thus throttling) to Microsoft API
refs INTEROP-6723
flag=microsoft_group_enrollments_syncing
Test plan:
- have a course with >= 3 members
- run a sync to make sure syncing still works
- change GET_GROUP_USERS_BATCH_SIZE in GraphServiceHelpers to 1000
The run in a console:
gsh = group.syncer_job.steps_object.graph_service_helpers
gsh.get_group_users_aad_ids(group.ms_group_id)
That should raise an error from Microsoft saying the batch size is too
high.
- Change GET_GROUP_USERS_BATCH_SIZE to 1, and add debugging to the line
before aad_ids.concat(...):
puts users.inspect
Then run in a console, as above
gsh = group.syncer_job.steps_object.graph_service_helpers
gsh.get_group_users_aad_ids(group.ms_group_id)
Users should be fetched in 3 batches of 1 user each.
Change-Id: I69f9bf260ad44078d329bfa1cd2d56ba6a99c54e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264928
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
I tried to keep this fairly simple. It's still more complicated than I
would like, and doesn't log everything perfectly, but I think it's a
good compromise. Note that this keeps the statsd metrics with tag
msft_endpoint: "post_$batch", but also adds a new class of statsd metric
with a new name (e.g. "microsoft_sync.graph_service.batch.error")
closes INTEROP-6772
flag=microsoft_group_enrollments_syncing
Test plan:
- open a console and monkey-patch statsd:
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD: #{args.to_json}"
end
end
- Run a synchronous sync (group.syncer_job.run_synchronously) that would
add or remove users and make sure the sync still works in general. If
removing users, you should see improved logs (e.g.
"MicrosoftSync::GraphClient: batch of 2 group_remove_users") and statsd
metrics (e.g. "microsoft_sync.graph_service.batch.success" with
msft_endpoint "group_remove_users" and status 204) for adding/removing
users. You can also see that I simplified the logging a bit to say
just "step ... finished with NextStep" instead of "step ... finished
with MicrosoftSync::StateMachineJob::NextStep".
- Try adding and removing batches of users from a group
that already has the users added/removed.
For instance, get some AAD ids from UserMapping and
get a group and run
gs = group.syncer_job.steps_object.graph_service
gs.add_users_to_group_ignore_duplicates(group.ms_group_id,
members: [aad1])
gs.remove_group_users_ignore_missing(group.ms_group_id,
members: [aad1])
Check for the new logging / stats as mentioned above. You should see
logs like this (note the 'ignored'):
DEBUG STATSD: ["microsoft_sync.graph_service.batch.ignored",2,
{"tags":{"msft_endpoint":"group_remove_users","status":404}}]
Change-Id: I7a1ba46b0e684c2e56f920e237f5a04778a32b7d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264804
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
According to the flakey builds, current_progress.results
was occassionally nil (though I rant them 300 times
locally and never saw an error).
I don't think it's worth testing insecure hosts at this
level since the logic is contained in canvas_http.rb and
adequately tested in canvas_http_spec.rb.
closes MAT-61
flag=none
test plan: jenkins build passes
Change-Id: Ic9178e9315da6e0f6bd3f258b45607b03fceaa33
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264954
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
QA-Review: Ed Schiebel <eschiebel@instructure.com>
Product-Review: Nate Armstrong <narmstrong@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
test plan
- see specs for functionality
- use address book with discussion_topic as context
- it should work
closes VICE-1429
flag=none
Change-Id: I3e0b773802725e3b70db3bb5b74af1325cf0ca8e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264736
Reviewed-by: Caleb Guanzon <cguanzon@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Caleb Guanzon <cguanzon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
I'm not 100% sure it makes a difference but this is safer.
Test plan:
- enqueue a syncer job (group.syncer_job.run_later)
- check that the job's strand has the global id
Change-Id: I6e55450d7cf707d50c5cde1a98fa83186a614723
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264207
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Microsoft UPNs are case-insensitive, and the Microsoft API may send back
UPN mappings in a different case from that which we requested them. When
that happens, we weren't using the UPN mapping returned. This fixes that
and also raises an error if we get some other UPN back from the API that
we don't expect (so would otherwise ignore), in case there is some other
similar issue.
closes INTEROP-6788
flag=microsoft_group_enrollments_syncing
Test plan:
- Make a new course with the following users enrolled. Note: it's
probably easiest to use users you've never used in Canvas before so you
can create users with email address with varying case easier. Look at
the Microsoft admin portal to see users. Canvas seems to find users
with an equivalent case-insensitive email, unless adding both users as
new users to the course at the same time.
- one user that is in our test tenant but has different casing,
e.g. INTEROP_Teacher@...
- two users that are in our test tenant but with different casing from
the test tenant and with each other, e.g. InterOp_STUDENT@ and
INTEROP_student@
- clear the UserMapping table
- run a sync
- check that the UserMapping table has entries for all three users, with
the users having the same (downcased) email addresses having the same
AAD id.
- check in Microsoft admin portal that both users were added to the
group (since it's case insensitive, users with the same casing will
only be there once of course)
Change-Id: Id0bf4cc3a7340715e4df930afcfb98ad83cb077a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264784
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
When looking up a user's UPN, we should only a user's verified email
addresses. This ensures that a user can't just add any email they want
and get added to course's Office 365 Group and Team, which could've lead
to some fun Teams meeting bombing.
closes INTEROP-6768
flag = microsoft_group_enrollments_syncing
Change-Id: If924f251b7e2898004edd91a9f3417887e3db41e
test-plan:
* Setup Microsoft Teams Sync for a course
* Ensure the course has a teacher and two students, all in our test
tenant, all with verified emails.
* Create a new user through the admin page, and make sure they use the
same email address as the teacher. Don't verify this email. (this
tests what happens if a student tries to masquerade as a teacher)
* Add the new user to your test course.
* Start the sync job, either through the UI (not sure if we have that
working yet) or by using a Rails console. For the console, grab the
Microsoft Sync group associated with the course and then run
"group.syncer_job.run_synchronously".
* Check and make sure that the new unverified student isn't added to the
Microsoft Team and group. If they are, then suddenly a student could
become an owner, and that would be really bad.
* Run through this same process, but have the new user try to use the
email address of one of the regular students. This tests what happens
if a student tries to masquerade as another student by adding the
other students email address.
Change-Id: If79dcedab7893d48eb27560e8bbc0c79f439ba8d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264690
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Tucker Mcknight <tmcknight@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
closes FOO-1761
flag = granular_permissions_manage_courses
test plan:
• ensure migration covers the proper role overrides to populate
any derived custome role types from the parent base role type
Change-Id: Ife2e1a7a4a3a1f4dcdddd6de69f72580620749ae
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261089
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
If something goes wrong with MS API while adding and removing members/
owners we'll stop scheduling the job to retry the same step again.
Changes:
* While adding members/owners and an errors as `already exist` was
thrown, we'll rescue the error, and dispatch a 'batch' request to
save each user individually, and then log the offenders;
* Batch "remove" requests. This was easy enough since we had to write
the batch code above, and I didn't want to write the ignoring code now
and haeve to redo it when we batch remove users. If any 404s are
returned (user is not a member/owner), we'll ignore it and log the
offenders. NOTE: the MS API does not distinguish between a group not
existing, a user not existing, and a user not being a member. We
assume it's the latter and ignore it. This shouldn't be a problem
because we shouldn't be needing to remove users from a just-created
group.
closes INTEROP-6718
closes INTEROP-6720
flag=microsoft_group_enrollments_syncing
Test plan:
Setup:
- Set up a sync so that it will add users to a group (for instance,
create a course with no group on the Microsoft side)
- Add a byebug in the step_execute_diff line 190;
- Add the following in the `run_batch` block in
`remove_group_users_ignore_missing`:
puts "DEBUG: #{resp.inspect}"
Testing additions w/ duplicates:
- Sync the course via rails console (group.syncer_job.run_synchronously)
- When byebug starts, goes to MS console admin and add the users
that would be added to sync.
- type 'c' to continue from the breakpoint
- Check that the `batch` endpoint was called, and the log has an warning
like 'Skipping add for 1: {"members":["..."]}';
Testing removals w/ missing users:
- Remove the users from Canvas course so that the next sync will remove
them from the group.
- Sync the course via rails console again;
- When byebug starts, goes to MS console admin and remove the
users and continue from the breakpoint
- Check the log has an warning 'Skipping remove for 1: ...';
- Check the output of the "DEBUG" above to see the response body when
the user has been removed already. There are two variants. One looks
like "One or more removed object references do not exist for..." (this
one happens if trying to remove *right* after removing from the UI).
One looks like "Resource ... does not exist or one of its queried
reference-property objects". This happens if trying to remove a bit
longer after removing from the UI, or the users were previously removed
with the API. Add the user to the group again in the MS UI and repeat
the last 3 steps to try to get both variants. If you can't, no
worries, I did when testing.
Testing multiple batches removals:
- In the Microsoft console, add at least 11-12 users as both members
and owners of a group. You don't have to have local Canvas users for
them.
- Remove the 'byebug'. Run the sync and make sure the extra members and
owners are removed. Watch in the logs that it makes two calls to the
'batch' endpoint.
Change-Id: Icdae9bf9ed371c54508283110cc2c32ee6055af5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263984
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
closes FOO-1940
flag=none
also read from vault via
vault client to make the cert
contents cachable and to avoid
having to check http status codes
from the naked HTTP get.
TEST PLAN:
1) run in beta.
2) Pulsar connection errors go away
or at least are no longer identifiably
tied to missing certs on disk
Change-Id: I079cfeb7d065c70b6b18b2ff5cf01d2760c1d3dc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264281
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
This way if users have not granted permissions for our app, or have
revoked permissions, the erros will not clog up Sentry or ErrorReports.
(We can still have an alert for a low percentage of 200s from the API to
catch the case where our creds are completely bad.)
refs INTEROP-6716
flag=microsoft_group_enrollments_syncing
Test plan:
- Given a MicrosoftSync::Group g, run:
ra = g.root_account
old_tenant = ra.settings[:microsoft_sync_tenant]
ra.settings[:microsoft_sync_tenant] = 'microsoft.onmicrosoft.com'
ra.save!
g.syncer_job.run_synchronously
- Check that the job stopes with graceful cancel error,
ApplicationNotAuthorizedForTenant. last_error should be set
in the group and workflow_state should be "errored" but should not
have been a stack trace on the console.
- Set the tenant back and run another sync just to make sure normal
syncs are still working:
ra.settings[:microsoft_sync_tenant] = old_tenant
ra.save!
g.syncer_job.run_synchronously
- The sync should run successully
Change-Id: I9c9323a53672f6247ca62db45cd158bb2751ca0b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263903
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
* Adds time metric for graph service calls. This can help us identify if
Microsoft's API is being slow.
* Adds status code as a tag, so we have more specific metrics than just
"error" (we also have the error metrics in StateMachineJob though which
may be more useful depending on what's going on)
* Makes a tenant not existing be a "Graceful Cancel Error" rather than an
unexpected error, as this likely means the user has not entered the
right tenant.
refs INTEROP-6716
flag=microsoft_group_enrollments_syncing
Test plan:
- Check that a sync still works
- change the delay in the DelayedNextSteps in SyncerSteps to something
small to make sure we see some 404s.
- open a console and monkey-patch statsd:
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD: #{args.to_json}"
end
end
- Run a sync and make sure statsd is getting a
"microsoft_sync.graph_service.error counter with "status_code" tag
equal to "404"
- Add a "raise 'foo" in request(), or disconnect from your wifi,
whatever is most convenient. Run a sync and check that statsd is getting
a "microsoft_sync.graph_service.error counter with "status_code" tag
equal to "unknown"
- run
MicrosoftSync::LoginService.new_token(OUR_TEST_TENANT)
Make sure a statsd metric with status_code 200 was incremented
- run
MicrosoftSync::LoginService.new_token('microsoft.onmicrosoft.com')
or some other tenant that exists but we don't have access to
You should get success (a 401 is generated later on if we use that).
Check that a statsd metric with status_code 200 was incremented
- run
MicrosoftSync::LoginService.new_token('!!!')
You should get a TenantDoesNotExist error and a statsd
metric with status_code 400 should be incremented.
- change the microsoft_sync_tenant in your root account config to
something that doesn't exist like in the last 2 steps. Run a sync job.
A TenantDoesNotExist error, but this should be a
GracefulCancelErrorMixin, so it should quit the job immediately and
set last_error on the group but not bubble up the error or send it to
CanvasErrors.
- I'm not sure how to test statsd time metrics without deploying. Specs
should probably be enough for that anyway
Change-Id: If9a3c053bb138075dc1fdc229a713dffb0a812b5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263859
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
closes INTEROP-6573
flag=microsoft_group_enrollments_syncing
test plan:
Jobs scheduling
* have a course with a msft sync group set up
* add a user to that course (ie, change or add an enrollment),
preferably a user that has a matching MSFT user (see MSFT tenant)
* find the sync job scheduled for 10 min in the future with:
`Delayed::Job.where("strand like ?","%enqueue_future_sync%").first`
* wait a minute
* add/remove another user to the same course
* that same job should now have a new run_at time, 10 minutes from the
second enrollment change
* repeat the same with a batch enrollment update, by adding a couple of
users to a specific section of that course, and then deleting the
section
* that same job should be scheduled for 10 minutes in the future
UI
* add/remove a user to that course
* go to the course settings page, and to the Integrations tab
* Status should say "Sync auto-scheduled"
* You should be able to manually sync
* Click Sync Now to manually start a sync job, confirm the UI behaves
as before
* add/remove another user to the course to auto-schedule another job
* Status should say "Sync auto-scheduled"
* You should not be able to manually sync, since the cooldown is still
in effect
Change-Id: Iec65e5bda63d38b701109350156d075b5065c34b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263485
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
Adds an include param to the submission api which includes the
read_state for each submission. If this param is included, the
associated submissions are then marked as read.
closes LS-2141
flag = canvas_for_elementary
Test plan:
- As a student, visit grades in a k5 subject course with some
assignments
- As a teacher, visit the gradebook for that course and change one
or two of that student's grades
- As the student, refresh the page and expect to see some blue dots
next to those assignments with new grades
Change-Id: Ia9d200a409f9af99abcd1c7a4260f673c8cfec3d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263882
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jonathan Guardado <jonathan.guardado@instructure.com>
QA-Review: Jonathan Guardado <jonathan.guardado@instructure.com>
Product-Review: Jackson Howe <jackson.howe@instructure.com>
closes INTEROP-6588
flag=microsoft_group_enrollments_syncing
test-plan:
* Have a course with ms student and ms teacher enrolled;
* Decrease the MAX_ENROLLMENT_MEMBERS to the amount of students and
teachers enrolled to that course minus one;
* Execute the group sync and notice the job will be cancelled gracefully;
* Check that group workflow_state=errored and the last_error matches
`A Microsoft 365 must have a maximum X members in a team`;
* Reset MAX_ENROLLMENT_MEMBERS to previous value;
* Decrease the MAX_ENROLLMENT_OWNERS to the amount of teachers enrolled
to that course minus one;
* Execute the group sync and notice the job will be cancelled gracefully;
* Check that group workflow_state=errored and the last_error matches
`A Microsoft 365 must have a maximum X owners in a team`;
* Reset MAX_ENROLLMENT_OWNERS to previous value;
* Execute the group sync and notice that everything still working as
expected;
Change-Id: I229b223da8a3c8866b7047e64e28ed19bb221879
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263562
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
test plan:
- render docs, and click "SIS CSV Format" in the left nav
- see the field name for homeroom_course under courses.csv
- ensure you can use this field to create a homeroom course or
update an existing course'd (with SIS ID) homeroom setting
flag = canvas_for_elementary
closes LS-2122
Change-Id: Ica7ceff2bdfb5035c15cf237cc230b6cd117a603
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263947
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Robin Kuss <rkuss@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Revert "turn down basic outcomes file uploads from 10 to 1"
Revert "switch transient attachment error in jobs to Delayed::RetriableError"
Revert "Retry file uploads for basic outcomes"
This reverts commit 0c73bbd428.
This reverts commit 90917e4830.
This reverts commit cec0fc7b06.
* this change caused 2 unforeseen issues:
* retrying expired google/office pre-signed S3 URLs 10 times is not a
transient error and gummed up job queues for days
* creating a submission hours or days before the attachment actually
gets the file data can create errored canvadocs, even when the file
is actually present
refs INTEROP-6749
flag=none
test plan:
* file upload in basic outcomes still works
Change-Id: If61318e7652e5eb673d0ca6d4b83ddd01dbad709
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264021
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
fixes DEMO-144
flag=none
test plan:
- Create a classic quiz assignment
- Update the submission_types field of the quiz
assignment via the REST API
(PUT /api/v1/courses/:course_id/assignments/:id)
- As a student take and submit the quiz
- Try updating the submission_types field of the quiz assignment
via the REST API. This time the field will not change
Change-Id: Ifc9237f27bcfdb1eb3107d0f948c351e81e5996f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257449
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Stephen Kacsmark <skacsmark@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
Product-Review: Stephen Kacsmark <skacsmark@instructure.com>
Change-Id: I5c1a9f260afc56887881f98cdab08ad3a25c45f3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263980
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
closes FOO-1860
flag=none
TEST PLAN:
1) invoke a producer creation
to send messages to the same topic
2) it should all be done via the same
actual producer object
Change-Id: I09f84b5db7c8b8ca8555dc5d7a6fabb5fbc6da82
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263941
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
* it's causing problems when google/office presigned S3 urls expire,
and then expire for all 10 attempts, and gum up the job queue,
since all 10 attempts need to run before passing to the next job
refs INTEROP-6749
flag=none
test plan:
* specs
Change-Id: Ic4f03d8da4f3918cd77e7184065682e07d570d83
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263950
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
Since we cache by full key anyway, there is no good reason
to convert it into a recursive hash only to convert it right back
Change-Id: Ifb356e38a2ac5e6e0477ebc5913e0d9128622489
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263932
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
* this prevents the need for checking for flag existence before calling
feature_enabled?, or the need to add real flags to the stubbed flags
in feature-flag-specific tests
refs INTEROP-6573
flag=none
[pin-commit-multiple_root_accounts=748b8bbe3d06c71f74e46e72bffd3fb17d388ea7]
test plan:
* specs
Change-Id: Ic6a7d32c1aea10671e6929f87202513e00fe8715
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263798
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Also, don't show job_state to the end-user, as this could have
[semi-]sensitive internal information (e.g. the retried_error field with
error message)
Note: I didn't bother setting last_error_report_id to null when the
error is cleared (job completion) because that doesn't affect the UI
anywhere and could theoretically be useful for debugging a sync that
failed but later succeeded. I'm open to changing that if anyone
disagrees and cares enough, though.
refs INTEROP-6716
flag=microsoft_group_enrollments_syncing
Test plan:
- add "raise SocketError" in any one of the syncer steps (first one will
do), reduce the retry timeouts (STANDARD_RETRY_DELAY) if you want, and
run a sync
- check the Course settings page: Under Integrations -> MicrosoftSync,
it should show the error:
* As Site Admin, it should show a link to an error report
* Masquerade as a teacher. it should not show a link to an error
report
* as the teacher, check the traffic coming back from the "show" endpoint.
It should not include job_state
- change the raise in the syncer steps to "raise StandardError" and run
the sync
- check the Course settings page and again, check that the link to the
error report shows up as an admin but not as a teacher
- remove the 'raise' and run a successful sync; check that no error is
shown in the UI
Change-Id: I9916d426d8c1e00342b0f29b450b5584ae1af57b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263645
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
and that we have a vault token
refs FOO-1842
flag=none
If a cert url is in the config, we need
to fetch and write it to disk before trying
to open a connection to pulsar or we'll
get failures.
Also add some config caching so
we aren't re-parsing the same YAML
blob over and over.
TEST PLAN:
1) specs should pass
2) be in a pulsar-enabled environment
with a cert configured
3) when you build a message_bus producer,
the cert is fetched and written to
disk at the configured path.
Change-Id: I2fb61badd05aedadb6d07475875692b30dd30db9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263687
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs INTEROP-6716
flag=microsoft_group_enrollments_syncing
Test plan:
This is a pretty simple change and GraphService has good specs. I tested
this out as follows, it's probably not necessary for someone else to unless they
really want to.
- open a console and monkey-patch statsd:
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD: #{args.to_json}"
end
end
- change delays in SyncerSteps to ensure 404s
- run sync and observe stats for "graph_service.success" and
"graph_service.notfound"
- change graph service 'request' to always throw a HTTPTooManyRequests
error
- run sync and observe stats for "graph_service.throttled"
- change graph service 'request' to always throw a HTTPBadRequest
error
- run sync and observe stats for "graph_service.error"
Change-Id: I3181857f2624609cf27083461d551d855a830b60
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263730
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
GraphService Statsd metrics: (success, failure, notfound) for
(method+first part of url).
StateMachineJob Statsd metrics: (complete, failure, final_retry, retry,
stalled).
all but complete have the step as a tag. all but complete & stalled have
an error class as a tag.
StateMachineJob sending to CanvasErrors, with level "error":
* final retry (under RetriesExhaustedError; should come through as
statsd metrics under RetriesExhaustedError AND the cause; see
cause_category in lib/canvas/error_stats.rb)
* unexpected errors (should come through as statsd metrics under the
original error, and the cause if there is one)
* job step doesn't match state (under InternalError)
refs INTEROP-6716
flag=microsoft_group_enrollments_syncing
Test plan:
- change the delays in DelayedNextStep.new(...) in syncer_steps to
almost nothing so you will for sure see a 404 when syncing
- open a console and monkey-patch statsd:
class << InstStatsd::Statsd
def increment(*args)
puts "DEBUG STATSD: #{args.to_json}"
end
end
- Make sure there is no group on the remote side and run a sync for a
new MicrosoftSync::Group. Run with `syncer_job.run_synchronously`.
- You should see statsd debug messages for:
* graph service success
* graph service notfound
* smj retry
* smj complete
- add a 'raise SocketError' into one of the steps so it will fail with
this retryable error. run the sync with the monkey patched statsd and
you should see:
* statsd DEBUG message for smj retry
* statsd DEBUG message for smj retry_final
* statsd DEBUG message for 'stats.error' -- one for
RetriesExhaustedError and one for SocketError
* Canvas::Errors log the error
- add a 'raise StandardError' into one of the steps so it will fail with
this retryable error. run the sync with the monkey patched statsd and
you should see:
* statsd DEBUG message for smj failure
* statsd DEBUG message for 'stats.error' for StandardError
* Canvas::Errors logs the error
Change-Id: Ic50cb2838875955c4378c61d4e3f27e5e71bb74b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263367
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
closes EVAL-1586
flag=annotated_document_submissions
Test Plan
- Have 2 students in a course.
- Create a manually posted, peer review, student annotation assignment.
- As Student 1 and Student 2, make some annotations and submit the
assignment.
- Assign Student 2 to peer review Student 1.
- As the teacher, leave some annotations and submission comments on
Student 1's submission.
- As Student 2, view Student 1's submission.
- Verify that Student 1's submission is viewable.
- Verify that Student 1's annotations appear to Student 2.
- Verify that the teacher's annotations and comments do not appear
for Student 2.
- Verify that the submission is not annotatable by Student 2.
- Leave a comment as Student 2.
- As the teacher, post the submissions.
- As Student 1, view the submission and verify that all annotations
and comments from previous steps are visible.
Change-Id: I00b3aab3bf22a05b7a4580ff5ddd7125cb8bace8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263286
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Kai Bjorkman <kbjorkman@instructure.com>
closes INTEROP-6637
flag=microsoft_group_enrollments_syncing
test-plan:
* Have a course with student and teacher enrolled;
* Have a Microsoft group recorded to this course;
* Keep sure these users have a CommunicationChannel and Pseudonym;
* To check the `email` login attribute I used the demo accounts:
demostudent and demoteacher;
* To check the `preferred_username` login attribute I used the interop
accounts: interop_student and interop_teacher;
* Helpers:
ID = 5
TENANT='sample'
def redo_g
MicrosoftSync::UserMapping.destroy_all
course = Course.find(ID)
group = MicrosoftSync::Group.find(ID)
gs = MicrosoftSync::GraphService.new(TENANT)
gs.request(:delete, "groups/#{group.ms_group_id}")
group.destroy!
group = MicrosoftSync::Group.find(ID)
group.restore!
group
end
def run_sync
group = MicrosoftSync::Group.find(ID)
group.syncer_job.run_synchronously
end
def run_later
group = MicrosoftSync::Group.find(ID)
group.syncer_job.run_later
end
def set_login_attribute_to(new_value)
group = MicrosoftSync::Group.find(ID)
root_account = group.root_account
root_account.settings[:microsoft_sync_login_attribute] = new_value
root_account.save
end
* Into rails console you should be able to change the login_attribute
and sync the team:
$ redo_g
$ set_login_attribute_to('email')
$ run_sync # or run_later
* You should be able to run the sync with `email` and
`preferred_username`;
* You should check in the microsoft admin that the team was created with
the correct `email` or `preferred_username` configured;
* I didn't test with `sis_user_id` login attribute and
`preferred_username` login attribute when it's a GUID;
Change-Id: I38eaf862bbcfb2e489a2c83f13cc985a4209b108
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263130
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
The activerecord query in ‘recently_missing_submissions‘
now uses a join between courses and enrollments to only
apply the condition to the right submissions.
fixes EVAL-1547
flag=none
test plan:
- Create a course (A) with at least one student enrolled.
- Create a course (B) with the same student enrolled.
- Create an assignment for both courses (A) and (B)
with a due date of one day before the current date.
- Conclude the enrollment for the student to course (B).
- Enable 'Automatically apply grade for missing submissions'
late policy for both courses (A) and (B).
- Wait 5 minutes to get the 'apply_missing_deductions'
cronjob executed.
- Notice that only the submission (A) is affected by the
late policy.
Change-Id: Ic8386f713117826f24dc414a6ff92f9565d09b1e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263274
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Adrian Packel <apackel@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
Give less confusing error when course owners do not have Microsoft users
associated with them. This has happened locally and been confusing, but
it will also give us a heads up if, for real customers, the way of
linking a Canvas user to Microsoft user is not working how they expect.
refs INTEROP-6556
flag=microsoft_group_enrollments_syncing
Test plan:
- Have a course with teachers but no users which correspond to users
in our test tenant,
- Run the sync. It should fail with an error saying 'A Microsoft 365
Group must have owners, and no users to the instructors of the Canvas
course could be found on the Microsoft side'. This can be seen on the
MicrosoftGroup in the last_error field.
- Run a full sync with valid teachers just to make sure it still works.
Change-Id: I9a8c4f4421d04ed4c17e541fffb2ce69154c9cdb
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263263
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Revert "avoid joining conferences to users unnecessarily"
This reverts commit d44d00d421.
Reason for revert: so turns out we do use the users in this json
response.
currently it is used in the front end when editing a conference
see #markInvitedUsers around
ui/features/conferences/backbone/views/EditConferenceView.coffee:199
test plan
- create a conference and add some users to it
- edit the conference and you should see the same users checked
Fixes VICE-1310
flag = none
Change-Id: I76765676faab1bcd19d3b13e0d10255fa6299167
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261960
Reviewed-by: Davis Hyer <dhyer@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
All the intermittent errors, anyway.
I may try to see if I can DRY up some of this later
refs INTEROP-6556
flag=microsoft_group_enrollments_syncing
Test plan:
- Change the delays in the DelayedNextStep to something very small (like
0.1) and run a new full sync. Watch the logging and make the sure the
retrying on 404 is still working. Run the sync all the way through
and make sure it makes the group and team successfully.
- Enqueue a job and then force one of the intermittent errors. Suggested
ways: disconnecting from wifi, or adding a raise in the code. Check
that errors are retried. Try when the job is in a couple different
steps.
Change-Id: I1dc9c5ef4c711dabf4ad349be60550de824f6d82
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263210
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
When we get a 404 in the execute_diff stage, go back to the
generate_diff stage.
Because we're now essentially retrying generate_diff & execute_diff as
one big step, it should just be one step, but we're going to add
separate progress-capture retry soon so we'd have to split it up again
then. And, testing them separately still in the meantime is nice. So it
was easier just to add "Retry with step".
That said, I've left this a separate commit in case we want to remove
support with "Retry with step" later.
closes INTEROP-6638
flag=microsoft_group_enrollments_syncing
Test plan:
- At the beginning of step_execute_diff, add:
if group.reload.job_state&.dig(:retries_by_step)&.dig('step_generate_diff').to_i < 3
raise MicrosoftSync::Errors::HTTPNotFound.new(
service: 'foo', response: Struct.new(:body, :code).new('foo', 404), tenant: 'foo'
)
end
- Run job synchronously (g.syncer_job.run_synchronously) and watch logs
to see that it retries 3 times, doing back to generate_diff from
execute_diff, sleeping the specified time for retry, before succeeding.
- Run job asynchronously (g.syncer_job.run_later) and check the same.
Change-Id: I366c959ec8198801788b8b55b58847fd7329c439
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263048
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
* Don't try to update group properties immediately after creation, since
it is likely to fail until the Microsoft API settles
* Don't try to create a team the immediately after group creation, since
it is also likely to fail until the Microsoft API creates
refs INTEROP-6638
flag=microsoft_group_enrollments_syncing
Test plan:
- Delete existing groups on Microsoft side and run a syncer job. Watch
logs to see that:
1. it enqueues a job for step_ensure_class_group_exists (delay of 2
seconds) before ever hitting the PATCH endpoint (you can see
endpoints hit with logs looking like "MicrosoftSync::GraphClient:
get ..."), and that when it does so, it updates the group to have
workflow_state=retrying and job_state with empty retries_by_step
2. it enqueues a job for step_create_team (delay of 10 seconds) before
ever hitting the endpoint to create the team (post
https://graph.microsoft.com/v1.0/teams)
- After it's done, check that the job completed successfully and a group
and team were created on the Microsoft side.
- From a console run
group.syncer_job.steps_object.step_check_team_exists(nil, nil)
again and make sure it returns COMPLETE.
- From a console run
group.syncer_job.steps_object.step_create_team(nil, nil)
again and make sure it returns COMPLETE.
- For completeness, run a sync job on the same group again to make sure
a sync after completion still works. This time,
step_update_group_with_course_data and step_create_team should never
run. If you want, you run the job synchronously to see the steps easier.
Change-Id: Ic2b697dd7cc0d1bbf75ad55d3593ee64975e3ab9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263012
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs INTEROP-6638
flag=microsoft_group_enrollments_syncing
Test plan:
- Run a new sync (make sure a group doesn't exist for your course on the
MS side, and use mygroup.syncer_job.run_later. NOTE: after deleting a
group it takes a couple seconds for this change to settle so you may
want to wait a few seconds after deleting so the new sync job doesn't
find the old group)
- While the job is sleeping/retrying, check the state and make sure
retries_by_step is being set for whichever step is is retrying on
- Edit GraphService#team_exists? and add
"raise MicrosoftSync::Errors::GroupHasNoOwners"
- Run another sync and make sure it is scheduling a retry for 30 seconds
then 90, then 270. (You should be able to watch the job logs and see
the run_at time for the job it schedules -- look for a blue INSERT
block with 'INSERT INTO "public"."delayed_jobs"', 'object:
!ruby/object:MicrosoftSync::StateMachineJob', and timestamps at the
end)
Change-Id: Ic7d7f69de9e24a140daf41c54a6409b0a58f5334
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262954
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
and abort the request
test plan:
* add something like `User.connection.execute("SELECT pg_sleep(30)")` to a
controller action like Login::CanvasController#new
* use curl to call it (`curl http://localhost:3000/login/canvas`)
* ctrl-c the curl request
* notice the server immediately "responds" with a 408, and check that
the query is no longer active in the db (with
`SELECT * FROM pg_stat_activity` and checking the `state` column;
the last query will still be `SELECT pg_sleep(30)`)
Change-Id: Iabd3268ef70e4f036ea1c0fc60865697b123a09a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261781
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
* Set last_synced_at after completion
* Rename cleanup_after_failure to after_failure for consistency with new
after_complete hook
* Don't enqueue if microsoft_sync_enabled is not set on the account
closes INTEROP-6726
Test plan:
- Set microsoft_sync_tenant on the course's root account but have
microsoft_sync_enabled as false
- Run sync (e.g. group.syncer_job.run_synchronously). Sync should quit
immediately before doing anything.
- Set microsoft_sync_enabled to true. Run sync. It should successfully
sync the course enrollments.
- Check that last_synced_at has been updated
Change-Id: I4c01104f2c1e74791553278f4db4bb33682c6f4d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262917
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
We're adding the https://purl.imsglobal.org/spec/lti/claim/lti1p1
migration claim that will support the `user_id` field.
closes INTEROP-6649
flag=none
test-plan:
* Have a LTI 1.3 tool installed;
* Launch the tool and verify the JWT contains the lti1p1 claim with the
user_id;
Change-Id: I9278a6b22c69336b8ef35190c09c171e9b62dce7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262421
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Microsoft API doesn't like it
Test plan:
- pretty trivial, specs are probably enough.
Change-Id: I1720cd9f0b78b7189a63c205c21bb5299d0840e6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262050
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
* Sets up framework for running a job with many steps, each step of
which can be retried
* If a step succeeds, the next step will run immediately and you can
pass in-memory data
* If a step fails with a retriable error, you can return a Retry object
which will instruct the framework to retry the same step, or fail if
the maximum number of retries has been surpassed. You can pass a block
to run only if a retry happens to store data for the retry. For
convenience, extra data can be passed directly to the retry (stored in
job_state[:data]).
* If a step fails with a non-retriable error, a cleanup function will
run and the job will be set to an errored state. This will bubble up /
produce a Delayed::Job::Failed (unless the error explicitly includes
GracefulCancelErrorMixin)
* Jobs run 1-at-a-time (per Group record) on a strand to prevent
interference from jobs running at the same time. If a new job
starts when an existing job is in-progress, it will be ignored, unless
the old job is considered to be stalled, in which case it will kick
off a new job.
* This commit also converts the Syncer to use this framework, and sets
up retry for creating a team, replacing the sleep. I will adjust the
exact timings later.
refs INTEROP-6571
flag=microsoft_group_enrollments_syncing
Note: the test plan below assumes the first time setting the properties
on a new group will fail, and after sleeping 45 seconds -- after
everything is consistent on the Microsoft side -- all subsequent
requests will succeed (except for possibly the team creation -- this is
retried with a 1 minute retry delay). However, I've seen widely varying
behavior in terms of what delays are necessary before which endpoints
will find the newly-created groups. You may see 404 errors still in
steps I don't have retry in yet. I'm working on adding retry in all
steps, in the meantime if you see 404s, let me know, but you can just
delete the group and try again.
Test plan:
- Make sure you have a course/group with a teacher and student
- Before each run/step below, delete the group on the Microsoft side
before syncing (in the admin console, or do
`gs.request(:delete, "groups/#{g.ms_group_id}")`and delete the
MicrosoftSync::Group record and create a new one for the course.
- Delete group & run synchronously with
`group.syncer_job.run_synchronously`
- During the 1 minute sleep, check the workflow_state and job_state of
the group. workflow_state should be retrying, job_state should have
{step: :step_ensure_team_exists, retries: 1, updated_at: ...}
- At the end, check in the Microsoft admin console that the group and
team have been created and the group has the teacher and student in.
- Delete group & run asynchronously with run_later, and with a jobs
container running.
Again, during 45-second or 1-minute wait period, check state of group.
- After it completes successfully, delete group on MS side and locally
again. Stop the job container. Then run the following:
c = Course.find(...)
g = MicrosoftSync::Group.create(course: c)
g.syncer_job.run_later
g.syncer_job.run_later
g.syncer_job.send(:run_with_delay, :execute_diff)
- Start the job container and wait until the job is in the retry wait
period (you will see a bunch of lines of yellow output with it
updating the microsoft_sync_group job_state). At that point you
should see the second job run and immediately quit immediately because
the group is in a retrying state. (Look for output like "Processing
MicrosoftSync::StateMachineJob#run(nil)" and "Completed
MicrosoftSync::StateMachineJob#run(nil)".) Then after that, you should
see the third job run (with argument :execute_diff) and fail with an
"InternalError" saying the step doesn't match the state.
- Try adding and removing some members/owners and without deleting the
MicrosoftSync::Group record or deleting the remote group, try syncing
again (with an async job) to make sure updates still work.
Change-Id: I8fafac65b79df8bf3ecea51d027424c1e3bda40e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262148
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
fixes INTEROP-6674
flag=none
Test plan
- Have an LTI 1.1 tool set up that allows grade passback
- Send a request to https://<canvas>/api/lti/v1/tools/:tool_id/grade_passback
with a bad file like this in your Canvas console or another with oauth
installed:
consumer = OAuth::Consumer.new(:tool.lti_key, :tool.lti_secret)
client = OAuth::AccessToken.new(consumer)
client.post(url, xml, 'Content-Type' => 'application/xml')
- The XML should look like below, but make sure to put in a url
and a sourcedId from a real assignment/user in your instance
- Ensure the job attempts to run multiple times
<?xml version = "1.0" encoding = "UTF-8"?>
<imsx_POXEnvelopeRequest xmlns="http://www.imsglobal.org/services/ltiv1p1/xsd/imsoms_v1p0">
<imsx_POXHeader>
<imsx_POXRequestHeaderInfo>
<imsx_version>V1.0</imsx_version>
<imsx_messageIdentifier>999999123</imsx_messageIdentifier>
</imsx_POXRequestHeaderInfo>
</imsx_POXHeader>
<imsx_POXBody>
<replaceResultRequest>
<resultRecord>
<sourcedGUID>
<sourcedId>3124567</sourcedId>
</sourcedGUID>
<result>
<resultScore>
<language>en</language>
<textString>0.92</textString>
</resultScore>
<!-- Added element -->
<resultData>
<downloadUrl>(url)</downloadUrl>
<documentName>(name)</documentName>
</resultData>
</result>
</resultRecord>
</replaceResultRequest>
</imsx_POXBody>
</imsx_POXEnvelopeRequest>
Change-Id: I60977230b15b945e592225dfca0f5057e960f924
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262457
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
This reverts commit 8704d83a37.
Reason for revert: Vault is not ready for prime time
Change-Id: I37c41be5345aa2ee362df4e7831cf64d2729668d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261952
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
fixes MAT-28
flag=none
test plan:
- Create a new page
- Switch to html view
- Add an instructure_file_link to an external file
<a class="instructure_file_link instructure_scribd_file"
href="https://google.com">google</a>
- Add an instructure_file_link to a mailto link
<a class="instructure_file_link instructure_scribd_file"
href="mailto:test@test.com">mailto</a>
- Add an instructure_file_link to a course file
<a class="instructure_file_link instructure_scribd_file"
href="/courses/1/files/1">file</a>
- Add an instructure_file_link to a course file download
<a class="instructure_file_link instructure_scribd_file"
href="/courses/1/files/1/download">download</a>
- Add an instructure_file_link to a course file preview
<a class="instructure_file_link instructure_scribd_file"
href="/courses/1/files/1/preview">preview</a>
- Add an instructure_file_link to a course navigation item
<a class="instructure_file_link instructure_scribd_file"
href="/courses/1/pages/a-page">page</a>
- Download button should only show for files
Change-Id: I97069ca5dfd8b539a4da2eb47fba48bdde594412
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262493
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
QA-Review: Ed Schiebel <eschiebel@instructure.com>
Product-Review: Nate Armstrong <narmstrong@instructure.com>
When launching a 1.3 tool as an unauthenticated user (possible in
public courses), Canvas sends the `sub` claim as
`https://canvas.instructure.com/public_user`
However, the sub claim should really be omitted when launching a tool as
an unauthenticated user, according to the IMS spec.
http://www.imsglobal.org/spec/lti/v1p3/#user-identity-claims
The `lti11_legacy_user_id` claim should follow the same behavior of
user_id laim in an LTI 1.1 tool. As `user_id` is empty when launching
a LTI 1.1 tool in a public course with unauthenticated user we're
changing the `lti11_legacy_user_id` claim to return an empty string
instead of `https://canvas.instructure.com/public_user`.
closes INTEROP-6599
flag=none
test-plan:
* Have a public course published;
* Install an LTI 1.3 tool with course navigation placement enabled, you
can use the this change in the LTI 1.3 Test Tool
https://gerrit.instructure.com/c/canvas-lms/+/262530, which disable
sub claim validation;
* As an unathenticated user, access the tool in the course navigation,
and verify that the tool should launch and the `sub` claim should not
be present and the `lti11_legacy_user_id` claim should em empty;
* As an athenticated user, access the tool in the course navigation,
and verify that the tool should launch and the `sub` and
`lti11_legacy_user_id` claims should be present;
Change-Id: I78bb64e3d898f44fcc401a43d054909032ef5420
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262530
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
refs FOO-1853
flag = none
test plan:
• tests pass
• spec is no longer flaky (verify via splunk)
Change-Id: I0907bb81c6d053caaccc88c1385b2d4f6c17f81a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262581
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
test plan:
* in console do something like `User.connection.execute("SELECT pg_sleep(30)")
* hit ctrl-c the curl
* check that the query is no longer active in the db (with
`SELECT * FROM pg_stat_activity` and checking the `state` column;
the last query will still be `SELECT pg_sleep(30)`)
Change-Id: I403e919423cf321e0f467727b077e3279f1ed6b8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262429
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Test plan:
- From a rails console, run:
MicrosoftSync::GraphService.new('ourtenant').list_education_classes
- There should be an error telling you where to put the creds
- Follow the instructions, hopefully they are self-explanatory.
- Run again and and the education classes should be listed
Change-Id: I4a8cf6f788178345bac375b8be6aa31b04bc4a24
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262423
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
In this case we don't care what arguments
ConfigFile#load is receiving, we just care that
the proper response is getting set for the method stub
Using .with can cause fragility if the expected args
don't match what the method stub is actually being
called with.
fixes FOO-1853
flag = none
test plan:
• tests pass
• spec is no longer flaky (verify via splunk)
Change-Id: I8800748d175aeadae6dccbcac011418679adde4e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262408
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
This commit also renames the InvalidStatusCode error and adds subclasses
for cases we semi-expect certain statuses.
The sleep is temporary; I am working right now on a framework to split
the job into steps with retry (and yet, run it synchronously with sleeps
if we want).
refs INTEROP-6571
flag=microsoft_group_enrollments_syncing
Test plan:
- Create a course with no enrollments
- Create the MicrosoftGroup and run the sync:
g = MicrosoftSync::Group.create!(course: mycourse)
g.sync!
- Check that the sync runs with no errors. Check in the
Microsoft admin console that a group was created for the course
- Add at least one teacher/TA/designer to the course
- Sync again. This time the team should be created.
Change-Id: If85ae3f1ad7f24a7e25f1811aca4e35043131326
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262074
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
closes FOO-1419
except for course copies
also had to loosen course copy and course audit records to allow for
course copies without a user (i.e. kicked off by the system)
test plan:
* create a template course, with some content, and assign it to an
account (you may need to do this in console)
* create a new course in that account via various methods
* the new course should always have content copied from the template,
unless you created it via a course copy
Change-Id: I9ec7e2e8dea47ee24aac43ee291c477d80424736
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260809
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
fixes LS-1984
flag=none
test plan:
- Make sure all the new file url rewriting features are enabled
and working.
Change-Id: I2cfa476e67cc0daa1d64debb45a5973c1aadedcf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261890
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Eric Saupe <eric.saupe@instructure.com>
refs INTEROP-6636
flag=none
* this commit should be merged/deployed last, and at least after
live-events-lti
test plan:
* with all related commits checked out,
* the live-events ecosystem still works
Change-Id: Ic91d46d4809e5118ec78dbb7c0895c3741ff5ada
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261318
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Xander Moffatt <xmoffatt@instructure.com>
Closes INTEROP-6648
flag=microsoft_group_enrollments_syncing
Test Plan
- Create a course with a name that contains
special characters and whitespace at the
start or end of the name.
- Kick off a MicrosoftSync::Syncer with that
course.
- When the syncer creatse the education class,
verify the mailNickname does not include the
special characters or whitespace.
- Verify the mailNickname does include the full
course UUID
Change-Id: I9f860c0b5ef97dd5971e70bc059d530ff134f977
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261553
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
When deciding whether to show instructor annotations in DocViewer for
moderated assignments, check whether the current submission has been
posted, instead of whether the assignment has had its grades published.
This ensures we don't show instructor annotations to students when the
instructor has clicked the "Release Grades" button on the moderation
page but not yet clicked the "Post to Students" button.
flag=none
fixes EVAL-1489
Test plan:
- Have an assignment with moderated grading and file upload submissions
- As a student (S1), upload something that is viewable in DocViewer
- As a provisional grader (PG1):
- Assign a grade to S1 and leave some annotations in DocViewer
- Assign some grades to any other students in the course
- As the moderator, open the moderation page for the assignment
- Select PG1's grade for S1, and whatever grades you like for other
students
- Click the "Release Grades" button, but do *not* click the "Post to
Students" button
- As S1:
- Open the submission details page for the assignment
- You should not see any annotations from PG1 (or anyone besides
yourself)
- As the moderator:
- Open the moderation page and click "Post To Students"
- As S1:
- Open the submission details page for the assignment
- You should now see PG1's annotations (as well as any annotations
left by the moderator)
Change-Id: Ic6b579ba832bb2d0d5ead8eb040872e963f31e71
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/262006
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Gary Mei <gmei@instructure.com>
QA-Review: Gary Mei <gmei@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
This sets the user filter sent to DocViewer to include more than just
the viewing user's id when the assignment is an Annotated Document.
closes EVAL-1553
flag=annotated_document_submissions
Test Plan
- Create an Annotated Document
- Set it to post manually.
- As a student, make some annotations and click Submit.
- As a teacher, navigate to that submission:
`/courses/:course_id/assignments/:assignment_id/submissions/:user_id`
- As a teacher, make some annotations. Refresh the page and verify
the annotations are still there.
- As the student, view that submission. Verify that the teacher
annotations do not appear.
- As the teacher, grade the submission and post it.
- As the student and the teacher, both users' annotations should be
visible.
Change-Id: I01bfbc36683ffc9120785df622b60ea98db3d913
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261978
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Syed Hussain <shussain@instructure.com>
Reviewed-by: Aaron Shafovaloff <ashafovaloff@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
Test plan:
- Make a course with a couple users (students, teachers, TAs) enrolled.
The users should have email addresses corresponding to users if our
Microsoft Tenant (check the Admin console if unsure).
- Clear out your UserMapping table, to make sure you test everything
- Make sure you have the test tenant set in your Account settings and
you have the creds loaded in dynamic_settings.yml
- Run the sync job:
course = Course.last # etc.
g = MicrosoftSync::Group.create(course: course)
g.sync!
- Use the Microsoft Admin console to check that the students are members
and the teachers/TAs/designers are members and owners.
- Remove a member on the Microsoft side, rerun the sync, and check that
it adds them back. Cases to check:
- Remove a non-owner group member
- Remove a teacher as a group member
- Remove a teacher as a group owner (or change to another user if it
doesn't let you delete the last owner)
- Remove a teacher as a group member and owner
- Add extra users on the Microsoft side as members and owners and make
sure they are removed.
- Ideally we could also try doing adding more than 20 users (or at
least n > 20, where n=2*owners+members), but we might not want to make
that many Microsoft users in the Admin yet (?)
- Sync with no changes necessary and watch for the logging that it
doesn't make any requests to modify the group (no requests to
`patch /v1.0/groups/...`)
refs INTEROP-6568
refs INTEROP-6570
flag=microsoft_group_enrollments_syncing
Change-Id: I930472cef7c2bbd7e2ecb7f57ac4b61245353809
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261379
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
refs QUIZ-8160
flag=none
test plan:
- test with quiz_lti patch /quiz_lti/+/261845
Change-Id: Ib8e1f8f2f5449b8ebbadea6067f8aaeef925c8f9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261846
Reviewed-by: James Logan <james.logan@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
Product-Review: Susan Sorensen <susan.sorensen@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
When a Course is concluded it shows the warning: Concluding the course
will archive all course content and prevent you from modifying grades.
That said, we'll raise an InvalidSourceId error when the user is trying
to grade passback a concluded course.
closes INTEROP-6438
flag=none
test-plan:
* Have a course recorded;
* Have a student and teacher enrolled in this course;
* Have an Assignment with external tool submission (LTI Outcome Service
Example), you can use the branch
https://github.com/wagnerjgoncalves/lti_example/tree/teacher_launch
which allow you to submit the score as a Teacher;
* Conclude the course;
* As a Teacher you should be able to launch the LTI Outcome Tool and
fill the sourceId and submit the score. You'll receive an XML with
codeMajor=failure and description=Course is concluded;
* If you un-conclude the Course and enroll the student again, you should
be able to score without any error;
* To recover the sourcedId, you can run this in rails console:
assignment = Assignment.find(ID)
user = User.find_by(name: NAME)
course = assignment.course
tool = assignment.external_tool_tag.content
payload = [tool.id, course.id, assignment.id, user.id].join('-')
sourcedid = "#{payload}-#{Canvas::Security.hmac_sha1(payload)}"
Change-Id: I50d753f95ec343d86e2f8ee31968bc55931f00a7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261875
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
closes EVAL-1515
flag=annotated_document_submissions
Test Plan
- Create an Annotated Document.
- As a student, annotate the document.
- Refresh the page and verify that annotating is still possible.
- Submit the assignment.
- View the Submission Details page.
- Verify that the annotations made previously appear.
- Verify that adding more annotations to that attempt is not
possible.
Change-Id: I6dbfdb3f107ef04e2a042d9c50bc719a1357a324
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261650
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Adrian Packel <apackel@instructure.com>
QA-Review: Gary Mei <gmei@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
fixes INTEROP-6426
flag=none
Test plan
- Set up the TurnItIn 1.1 LTI tool on an assignment
- Upload a document to their tool as a student
- Send a request to get the TurnItIn submission
download process started (see below)
- Try to get DocViewer annotations working
and add some
- Resend the request to get the TurnItIn submission
- Ensure you don't lose the annotations
EX: In IRB fill out the following:
require 'json'
require 'oauth'
oauth_key = (from your tool consumer key)
oauth_secret = (from your tool shared secret)
paperid = (after you've uploaded a document via
the LTI tool, click on it and use the o parameter
from the URL)
outcomes_tool_placement_url = (https://sandbox.turnitin.com/
api/lti/1p0/outcome_tool_data/#{paperid}?lang=en_us)
sourcedid = (from the LTI launch the value in
lis_result_sourcedid)
tool_id = string tool id from the tool
post_url = (from the LTI launch the value in
ext_outcomes_tool_placement_url)
json = {"outcomes_tool_placement_url"=>outcomes_tool_placement_url,
"paperid"=>paperid,
"lis_result_sourcedid"=>sourcedid,
"tool_id"=>tool_id}.to_json
consumer = OAuth::Consumer.new(oauth_key, oauth_secret)
token = OAuth::AccessToken.new(consumer)
response = token.post(post_url, json, 'Content-Type' => 'application/json')
This starts a delayed job that checks with TurnItIn for the score.
If done right, it should not take long.
Change-Id: I80ac8bd666a25b8b52ace46befe66daaad32b2c4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261550
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
Add Canvas.assignment.submission.studentAttempts and
Canvas.assignment.allowedAttempts variables.
Fix `GAURD` typo.
closes INTEROP-6590
flag=none
test-plan:
* Have a course recorded;
* Have a student enrolled to this course;
* Enable `Assignment Allowed Attempts` in the course features options;
* Create an Assignment with online submission and fill the `Number of
Attempts`;
* Logged as student, you should be able to submit your homework until
the max allowed attempts;
* Change the assignment submission type to external tool, you ca use
the LTI test tool, in the custom params field fill the new custom
variables:
{
"allowed_attempts": "$Canvas.assignment.allowedAttempts",
"student_attempts": "$Canvas.assignment.submission.studentAttempts"
}
* Launch the tool as a student, you should be able to see the two new
variables expanded;
* Launch the tool as a teacher, you should be able to see only the
allowed attempts variable expanded;
Change-Id: I23a5d38f69a94279a2e73d169e1e274659dc5d0f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261699
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
test plan
- specs should pass
- the root_account_ids should adding users via account admin page
flag= none
refs VICE-1260
Change-Id: I5440fbc24959844520b429f6b534b3a56c9f3996
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261830
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Matthew Lemon <mlemon@instructure.com>
QA-Review: Matthew Lemon <mlemon@instructure.com>
Product-Review: Matthew Lemon <mlemon@instructure.com>
we don't show the user ids belonging to a conference in the UI anywhere,
so we shouldn't need to load it.
closes FOO-1791
flag = none
test plan: basic regression test of conferences UI.
Change-Id: I62753edee5d43b4c514c35ec14cb6cc58a2d7fb2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261782
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
also moves the RootAccountResolver specs out of enrollment_state_spec.rb
and into a more generic root_account_resolver_spec.rb
refs FOO-1171
flag=none
test plan:
- migration works up and down
- tests pass
Change-Id: Ia3ee89d63974e4c787ca8d1c6ab80eba5f54beb4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260526
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
refs FOO-1171
flag=none
test plan:
- migration works up and down
- tests pass
Change-Id: I944d86adc191e5cb18a2129546e1661ae6197b78
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260529
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
closes OUT-4013
flag=improved_outcomes_management
Test Plan:
- Make sure you have a course or account with outcomes
- Make some queries in /graphiql and check if search is returning
results related to your search:
t push gerrit
Query:
{
course(id: COURSE_ID) {
rootOutcomeGroup {
outcomes(searchQuery: "MY QUERY HERE") {
nodes {
... on LearningOutcome {
title
description
}
}
}
}
}
}
Change-Id: I6c50385ed0fe36c5b9e75a831edddeaf51561c18
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259826
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
QA-Review: Pablo Gomez <pablo.gomez@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
I added two tickets for further work:
- INTEROP-6637: Supports non-email address UPNs.
- INTEROP-6640: correctly/efficiently handle multiple email addresses.
Need to think a bit on how best to do that.
- INTEROP-6641 (and INTEROP-6608): retry & retry based on throttling
closes INTEROP-6569
flag=microsoft_group_enrollments_syncing
Test plan:
- Add some users in your test course with email addresses that
correspond to the users we have accounts for in our microsoft tenant
(talk to me if you are unsure of what these are)
- Run Syncer#sync for a Microsoft Group for the course
- Check that `UserMapping`s have been created corresponding to the user
and the AAD object ids for their email addresses (look at the
Microsoft admin console for our test tenant to get the IDs, or look up
the IDs from the API with `graph_service.list_users`)
- Run the job again. Test both of these cases:
- If you have extra users enrolled in the course that don't have email
addresses or don't have users on the Microsoft side, you should see
the job hitting the MS API again for them, but no new UserMappings
should be created.
- If you don't have any extra users, you should see that the the job
doesn't the MS API.
- Clear out your UserMappings, unenroll one user (enrollment should have
a workflow_state of 'deleted') and sync again. It should ignore the
deleted enrollment.
Change-Id: I58575db23de7b2b56c785784efc53b8bd6b8216a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261093
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
* Lays out framework for getting and caching Microsoft API tokens and
hitting the Microsoft Graph API.
* Simple sync job which creates a group for the class if it doesn't
exist and sets the extra LMS/SIS metadata.
NOTE: I discovered the MS API is eventually consistent -- it does not
immediately create a group when we hit the POST /education/classes
endpoint. In a future commit (I added a new ticket) we can work replace
the "sleep" with polling with sleeping or launching a new delayed job.
closes INTEROP-6567
flag=microsoft_group_enrollments_syncing
Test plan:
* Get creds from me and put in dynamic_settings.yml
* Try getting a token and observe it is cached (only the first time you
run will you see "get service:timeouts:microsoft_sync_login:error_count"
and "set microsoft_sync_login/<OURTESTTENANT>" calls to the redis cache):
MicrosoftSync::LoginService.token(<OURTESTTENANT>)
* Put our test tenant (you can get that from me) into your root
account's settings[:microsoft_sync_tenant]
* Create a course with a name and description that clearly label it
as your test course for the Microsoft sync project
* From a rails console run:
c = Course.find(...) # your new test course
g = MicrosoftSync::Group.create(course: c)
syncer = MicrosoftSync::Syncer.new(g)
syncer.sync
* Run the syncer again and observe (from the MicrosoftSync::GraphClient:
log lines) that we do not create or delete anything.
* Check that the group looks OK on the Microsoft side:
g = MicrosoftSync::Group.last
syncer = MicrosoftSync::Syncer.new(MicrosoftSync::Group.last)
cgs = syncer.canvas_graph_service
gs = cgs.graph_service
gs.get_group(g.ms_group_id)
* Check that the extra properties on the group have been set:
gs.get_group(g.ms_group_id, select:
%w[microsoft_EducationClassLmsExt microsoft_EducationClassSisExt])
* Update the group's ms_group_id to nil in the database:
g.update! ms_group_id: nil
Then rename the course to something similar but slightly different and
re-rerun the sync job (you may have to run syncer.course.reload and/or
syncer.group.reload to make the syncer get new values). Then repeat
the last two steps. In the first of those two steps, the group name,
but in the second step, the data in microsoft_EducationClassLmsExt
should be updated. This is because the syncer assumes if a group has
been created it was created correctly, but it doesn't assume the extra
metadata was ever set to it re-sets it. Also check that ms_group_id
has been re-filled in on the group record.
* Change your creds to something invalid and clear the rails cache.
Run the sync. It should raise an error. Then check that the Group
record has been updated with workflow_state=errored and
last_error="Invalid Status Code: Login service returned 401 for tenant
<OURTENANT>". This is what will be shown to the user (teacher).
* If you need/want to look at the Microsoft admin web console, you can
get those creds from me. From there you can delete your group, if
needed. You can also delete your group by using:
gs.request(:delete, "groups/#{ms_group_id}")
Change-Id: I077497c1c412095420079ea80d461bd172425bca
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260232
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
refs OUT-4247
Test plan:
- ensure results from aligned rubrics and from quizzes
are visible in the learning mastery gradebook
Change-Id: Ie9731f2c702f91b3da2ce4d8ae43f7045ee5509e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260613
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
refs FOO-1171
flag=none
test plan:
- migration works up and down
- tests pass
Change-Id: Ib309f1249a84d44492af8298d44fee3efa40992c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260534
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
there is an index on the user_id and context_id, we want to find from
the keys of the index and let root_account get populated from favorite
context
test plan
- specs should pass
fixes VICE-1236
flag=none
Change-Id: I70f2ceaa941d38cd7a7de054b0888044d3325a25
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261298
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Matthew Lemon <mlemon@instructure.com>
QA-Review: Matthew Lemon <mlemon@instructure.com>
Product-Review: Matthew Lemon <mlemon@instructure.com>
The `infer_file_extension` first should try to infer the extension from
name, filename, or url parameters, because, there's more than one mime
type to the same extension, like text/plain: dat, txt, hh, hlp.
In case the extension wasn't find, we will try to find using the
content_type parameter.
closes INTEROP-6616
flag=none
test-plan:
1. Have a Course recorded;
2. Have a Student enrolled in this Course;
3. Have an Assignment recorded to this Course with some restrict upload
file types;
4. Enable the assignment enhancements feature in the course settings;
5. Logged as a student, you should be able to upload files with
diferrent extensions (according to extensions restricted in step 3)
without receiving the console error `filetype not allowed`;
6. You should be able to receive the console error `filetype not allowed`
when trying to upload a file with a extension that wasn't in the
restricted list;
7. Accessing the submission details, you should be able to validate if
the attachment corresponds to the file you submitted;
8. Accessing the speed grader as a Teacher, you should be able to preview
or download the file submitted;
9. You should be able to you should be able to upload files after
disabling the assignment enhancements feature and execute steps 5
until 8;
Change-Id: Ie54b16fb3b6906554a2e1aad6010ad54d86a60f3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260893
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
load the context's shard before looking up past_lti_ids
test plan:
• specs pass
• verify in splunk after merge the test is no longer flaky
Change-Id: I5ee1967f17b5396254c6ea5567742eaaf57ad8b4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/261169
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
fixes FOO-1591
flag = none
test plan:
- have two users observing the same student
- remove user 1 as an observer
- merge user 2 into user 1
- it should work
Change-Id: Ic7cecc446e88a62916329d0411fa49d0787f89a1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258572
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
no method error was throwed when a unexpected context was passed
thru a polymorphic_url for a Message object
Test Plan:
- Monitor the sentry logs to check if theres no new occurencies
fixes VICE-1190
flag = none
qa risk=medium: It may causes unexpected behavior when anchor media
links aren't proccessed as expected due to this unexpected parameter
bug
Change-Id: I0477e0337682f99c84575fb1245bc19a58604cf8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260911
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Fixes INTEROP-6627
flag=none
Test Plan:
- Install an LTI 1.3 tool that uses
custom params + deep linking + https
- Embed a link from that tool in an RCE
or assignment
- Attempt to copy the course and verify
the resource link is copied to the source
course
Change-Id: If521039d30a9be33a8b6fb3688790f4013a2135d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260972
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ethan Vizitei <evizitei@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Updates Outcomes::LearningOutcomeGroupChildren by adding cache
at the queries for getting the total subgroups and outcomes.
closes OUT-4148
flag=improved_outcomes_management
Test plan:
- Create nested learning outcome groups
- For each nested learning outcome group create learning outcomes
> With FF improved_outcomes_management: OFF
- On Rails console: calls to Outcomes::LearningOutcomeGroupChildren
methods should return a default value
- On web: when generating actions over ContentTag, LearningOutcome
and LearningOutcomeGroup it should not lead to clear any cache
> With FF improved_outcomes_management: ON
- On Rails console: call Outcomes::LearningOutcomeGroupChildren
methods for getting total subgroups and outcomes, queries to the
DB should be made (it will need the root context)
- Call again the same methods, it should return the values from
cache
- Create a new instance of the class and call the same methods,
it should return the values from cache
- Clear the cache with `Rails.cache.clear`
- On web (or through GraphiQL) get the total subgroups and total
outcomes multiple times; it should cache the data and should not
run additional queries
- On web: clear the cache by executing the following actions:
- Add a Learning Outcome Group
- Adopt a Learning Outcome Group
- Copy a Learning Outcome Group from global
- Remove a Learning Outcome Group
- Add an Outcome
- Remove an Outcome
- Get the total subgroups and total outcomes, it should run new
queries
- Run the same tests for global context, it should generate and
clear the cache in the same way
Change-Id: I9b0bfc68b84b3e36869d69a926ef84d9989ea96d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257257
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Chrystal Langston <chrystal.langston@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
refs FOO-1171
flag=none
test plan:
- migration works up and down
- tests pass
Change-Id: I027da73a5ff87226206e3ba7b06618c0919ad8d1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260531
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
refs FOO-1171
flag=none
test plan:
- migration works up and down
- tests pass
Change-Id: Ib1e7f0cd2398c56fff8ec0208ace8a69a180a393
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260522
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
refs FOO-1171
flag=none
also don't backfill nulls if the column is already non-nullable
test plan:
- migration works up and down
- tests pass
Change-Id: I5009a2f31249643738f69a6b0d765daf89276165
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260519
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
the ContextModuleProgression may be saved more than once while
requirements are evaluated, and these saves may generate spurious
course completion live events based on incomplete data. evaluating
an item requirement may also trigger delayed re-evaluation of other
requirements.
give these changes time to settle before sending any course progress
live events. queue a singleton job to do so, so multiple changes
in close succession do not result in multiple live events. the
delay is configurable as the "course_progress_live_event_delay_seconds"
Setting (default 120, two minutes).
test plan:
- enable live events in a production-like environment
- have a set of modules with requirements
- as a student, complete some of them in rapid succession
- wait two minutes, and a course progress live event should be
created and up-to-date when received (counting the right
number of completed requirements)
- as the student, finish the requirements
- two minutes later, a course completion live event should be
received
alternatively you could modify the Setting and customize the
expected delay
closes LS-1967
Change-Id: I5c5ab01e75c48cc9916d9299ee1be55d360d1ee7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260416
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
closes INTEROP-6605
flag=none
When a user was submitting a zip file with `application/x-zip-compressed`
extension, we weren't accepting and returning `unable to find extension`
error.
So, we're changing the `canvas_mimetype_fo` gem to work with multiple
mime_type for the same extension like zip: application/zip,
application/zip-compressed, and application/x-zip-compressed.
And changing the `infer_file_extension` method to try to find the
extension from the filename when we don't know the content_type given.
test-plan:
test-plan
* Have a Course recorded;
* Have a Student enrolled in this Course;
* Have an Assignment recorded to this Course;
* In the course settings, you have to allow the assignment enhancements
feature;
* Logged as a student, you should be able to submit the homework by
uploading a zip file with different mime types like `application/zip`
and `application/x-zip-compressed`;
* Accessing rails console, you should be able to validate if the
Attachment.content_type corresponds to the file you submitted;
* To upload a zip file with `application/x-zip-compressed` mime type I
used parallels to have a Windows running in Mac;
* You should be able to submit the homework after disabling the
assignment enhancements feature;
Change-Id: If3a0dd3da67072c6292e65c551d83be9e235e7f2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260472
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
test plan
- specs should pass
fixes VICE-1004
flag = reduce_push_notifications
Change-Id: If227c3be628ac4a225f56dd91acc09d12c45a07c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260087
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Davis Hyer <dhyer@instructure.com>
QA-Review: Davis Hyer <dhyer@instructure.com>
Product-Review: Davis Hyer <dhyer@instructure.com>
fixes INTEROP-6185
flag=none
Test plan
- Set up the TurnItIn 1.1 LTI tool on an assignment
with limited attempts (there's a feature option
you have to turn on) so you can see the attempts
for the user
- Upload a document to their tool as a student
- Send a request to get the TurnItIn submission
download process started (see below)
- As a student, check your submission page and
see what attempt # it shows you
EX: In IRB fill out the following:
require 'json'
require 'oauth'
oauth_key = (from your tool consumer key)
oauth_secret = (from your tool shared secret)
paperid = (after you've uploaded a document via
the LTI tool, click on it and use the o parameter
from the URL)
outcomes_tool_placement_url = (https://sandbox.turnitin.com/
api/lti/1p0/outcome_tool_data/#{paperid}?lang=en_us)
sourcedid = (from the LTI launch the value in
lis_result_sourcedid)
tool_id = string tool id from the tool
post_url = (from the LTI launch the value in
ext_outcomes_tool_placement_url)
json = {"outcomes_tool_placement_url"=>outcomes_tool_placement_url,
"paperid"=>paperid,
"lis_result_sourcedid"=>sourcedid,
"tool_id"=>tool_id}.to_json
consumer = OAuth::Consumer.new(oauth_key, oauth_secret)
token = OAuth::AccessToken.new(consumer)
response = token.post(post_url, json, 'Content-Type' => 'application/json')
This starts a delayed job that checks with TurnItIn for the score.
If done right, it should not take long.
Change-Id: I6fd939eae77aaf654f271c947016c3d47176fbb1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260294
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs FOO-1644
[fsc-timeout=30]
- declare dependency on dynamic_settings
- move Auditors code over to Audits module
- declare dependency on canvas_cassandra
- declare dependency on event_stream
- get tests on Auditors module from parent app to pass locally
- shim Auditors to point to Engine
- update in-repo callsites to use module from engine
Change-Id: I78aaf27c236bcb3a4cd32b3773ec7acddc30cd39
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259222
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
apparently this tag isn't allowed to self-close
test plan:
- record a video in course content and new RCE
- add some text below the video
- export the course to a package and import the package
- the text in the copy of the item should still be there
(specifically, it should still be outside the iframe,
not inside it where it's invisible)
flag = none
fixes LS-1968
Change-Id: I1e189e850fa261059e8b6976947dc18320b49a25
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260210
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
refs FOO-1648
flag=none
move general module code to "gems"
along with specs.
Leave shim in canvas to avoid breaking
things while callsites get changed.
change some limited callsites
from Canvas::Errors to CanvasErrors
that were doing spec stubbing that required
at least one namespace change anyway
generate readme describing common use
cases.
TEST PLAN:
1) we still send stuff to sentry
Change-Id: I0111afc21e9e8c6604370885b4c750186b96175f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260035
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
when the transaction was committing, it wasn't rolling back
Change-Id: Ie75b7dc867c64da527bf99493059c250d08bc153
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259779
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs FOO-1648
flag=none
[fsc-propagate=false]
move general module code to "gems"
along with specs.
Leave shim in canvas to avoid breaking
things while callsites get changed.
change some limited callsites
from Canvas::Security to CanvasSecurity
that were doing spec stubbing that required
at least one namespace change anyway
generate readme describing common use
cases.
TEST PLAN:
1) security operations continue unimpeded.
Change-Id: Ia2d102d5038e2f5d0bb24201e38894e12b73063e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259540
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1636
flag=none
[fsc-propagate=false]
- add gem initializer to canvas for canvas_cache
- move Canvas::Redis into canvas_cache
- alias module in canvas
- expand dependencies of canvas_cache
- add redis dependency to vendored gem environment
- extract RedisConnections to deal with disconnect/clear_idle
TEST PLAN:
1) continue to cache things in redis
2) caching behavior with canvas-extensions
should not change
Change-Id: I96383eced80d670e14e535993f0c0608ab5f6520
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259434
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs INTEROP-6502
flag=none
* Have an LTI 1.3 installed;
* Have a Course recorded;
* Have an Assignment with lti resource links recorded in the `Course`
context, use the RCE editor placement;
* Have an Assignment with lti resource link recorded in the `Assignment`
context, use External Tool Submission;
* You sould be able to create a new course or use someone available;
* Into course settings, you sould be able to `Import Course Content` by
choosing an option from the `Content type` dropdown menu;
* When the job is completed, you should be able to check if the resource
links were created properly using rails console:
$ Course.find(ID).assignments.map(&:lti_resource_links)
$ Course.find(ID).lti_resource_links
and you should check if the new assignments were launching the tool and
expanding the custom params as expected;
Change-Id: I9ffc71a832409e6cfd488822f1e39d5c129bfff8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258967
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
refs FOO-1615
flag=none
first step of verifying modularization path for
chunks of behavior inside canvas.
take a dependency on config_file to minimize
unnecessary injection
move initialization of dynamic_settings to one place.
leave pointer in previous initializer file.
TEST PLAN:
1) nothing changes at runtime
2) dynamic settings specs can be run
quickly on their own
Change-Id: I1bfbd5b8dfedb9835953d9fc663259c17b76ce7f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258762
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1648
flag=none
These login audits are a group of functionality
very specific to the pseudonym login case,
and they know a few things about the pseudonym model.
Moving it into it's own module makes sense from
and organizational standpoint, but also makes it
more reasonable to extract the rest of canvas_security
into a gem for other gems/engines to depend on.
TEST PLAN:
1) the pseudonym login process remains unchanged
2) specs pass
Change-Id: I8f689c9255c33f0ac4d7947e3b0a0592d7900463
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259557
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
refs FOO-1648
flag=none
Canvas::Security depended on
ServicesJwt for secret loading.
ServicesJwt depended on Canvas::Security
for decoding and decryption
pushing the secret loading into the
security module breaks the circle
so we can extract canvas_security
without having to bring everything
at once.
TEST PLAN:
1) JWTs continue to get decrypted as they have
2) specs pass
Change-Id: I08c42b9727bee3660998cc2b0d9139beb496e244
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259550
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs FOO-1648
This method doesn't belong in core
security because of all it's dependencies
on models and PluginSettings. These are circular
because models depend on Canvas::Security.
We need to move it so we can extract the rest
of canvas_security safely.
TEST PLAN:
1) mass re-encryption works as expected
2) specs pass
Change-Id: I1e28d83b09e36e9d72408a7dbd7276329162434b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259545
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs FOO-1632
[fsc-propagate=false]
leave shim for plugins, but change
all in-repo callsites to gem version
of constant
TEST PLAN:
1) nothing changes
2) workloads talking to cassandra still converse with
the correct databases.
Change-Id: I0ec6bbbee3a298762e6dce937f3b8cb690b474ad
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259318
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
refs FOO-1636
flag=none
this helps ease the migration of caching
functionality into the canvas_cache
gem by unwinding circular references between
these modules.
Canvas now invokes Canvas::Redis, but
Canvas::Redis no longer invokes methods
on Canvas.
TEST PLAN
1) caching continues to work the same in operational environments
Change-Id: I99a55a63def4fd01e6195f26ec69ae8f62d76314
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259424
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
If a course with final grade override enabled imports a CSV with no
override score columns, don't bother querying for existing override
scores at all. This fixes a crash that could occur when calling
find_each on a "none" scope.
fixes EVAL-1502
flag=import_override_scores_in_gradebook
Test plan:
- Have a course with final grade override enabled
- Enable the "Import Override Scores in Gradebook" site-admin flag
- Import a CSV without any "Override Score"
- The import should complete and should not throw an "invalid CSV file"
error
- Try importing a CSV with at least one "Override Score" column for good
measure
Change-Id: Id0cf490c5862144b5a67524156ea440b4514c0c5
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259397
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Syed Hussain <shussain@instructure.com>
Reviewed-by: Gary Mei <gmei@instructure.com>
Reviewed-by: Spencer Olson <solson@instructure.com>
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Syed Hussain <shussain@instructure.com>
refs FOO-1627
flag=none
make Dockerfile cache use
gem instead
TEST PLAN:
1) specs run successfully
2) config in operational environments loads successfully
3) reloading settings still causes config files to get
re-parsed
Change-Id: I69bdfc2b79065600336f83c7ffb87dd8e647fa9c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/259232
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
closes INTEROP-6321
flag=none
Expected import process:
1. Import assignments as usual. Allow them to
create default line items and RLIDs, but
use the lookup_id from the assignment settings
export.
2. Import resource links. If a link already exists
with the given lookup_id, update the existing link
(which should belong to an assignment) with the
custom params of the given resource link
Test Plan:
1. Create an LTI 1.3 assignment with custom params
2. Create a few links in an RCE with via deep
linking with custom params
3. In the same course, navigate to settings and
choose "Export Course Content"
4. Once the package is build, download and extract
it.
5. Open imsmanifest.xml and verify that each
Lti::ResourceLink created in steps 1 and 2
have a `resource` element with type
`imsbasiclti_xmlv1p3`
For each of those resource elements, open the
file pointed to by the child `file` element
and verify the files contain the correct
lookup_id and custom pramas
6. Verify course exports in a course with no lti
resource links continues to work without error
Change-Id: I0092a5c9c6ec98b7566bc1a70dd0a030b0043032
Change-Id: I1699a4d291623d89875bfdad0f5d04dfb7845c16
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258005
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
closes OUT-4045
flag=none
Test plan:
- in course files, create three folders, each containing an image
- also add three images not in folders
- unpublish one folder and one of the top level files
- set access to one of the files and one of the folders to unlock
in several minutes and lock again a few minutes later
- create a quiz in the course
- add a quiz question
- add the six images to the text of the quiz question
- save the quiz
- take the quiz as a student (test student works)
- verify that four of the images are not visible to the student
- go to course files and publish the hidden folder and file
- take the quiz again as a student
- verify that the two unlocked images are now visible
- wait until after the unlock time set above is reached
- take the quiz again as student
- verify that all the images are now visible
- wait until after the lock time set above is reached
- take the quiz again
- verify that the lock_at folder and file images are now
not accessible
Change-Id: I1e7cb7c103158c0ef5e7704459970554c59cd7e9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257917
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
it's only public sometimes because of how it's been extended
Change-Id: I64793fe54a6a98e7b509f968bf53aa6ba349f6fd
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258931
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[fsc-max-nodes=12]
[fsc-timeout=60]
* switch lots of parsing to Nokogumbo to keep things consistent
* deep CSS sanitization is now built in, and with a proper parser (meaning
we can drop our code to do it, and adjust some specs to account for things
that _are_ valid)
lots of changes because gumbo parsing<->serialization cycle is slightly different:
* better job preserving original whitespace
* literal non-breaking space characters are converted to entities
* <p> tags aren't inserted for the heck of it
* several _other_ entities are unnecessary, and output as literal characters
* some elements no longer have a closing tag
Change-Id: I7c5e36cbd04b8a05f64c9e0af00868dd6b00f4ce
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256444
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
This is part 3.1 of changing the datatype from varchar to UUID of
lookup_id and resource_link_id from lti_resource_links.
Stops writing into lookup_id and resource_link_id columns.
refs INTEROP-6488
flag=none
test-plan:
* specs should pass;
* you should check if LTI is launching as expected, and if the custom
params was expanded as expected in all records that were created in
the part 1, 2 and 3;
* you should be able to new persist custom params, for example you can
use the RCE editor placement;
* you can follow the test-plan:
* https://gerrit.instructure.com/c/canvas-lms/+/256029
* https://gerrit.instructure.com/c/canvas-lms/+/254453
Change-Id: I9b2c85e0afb333c17afb1be0fb4a543f145ba400
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258317
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
fixes INTEROP-6525
flag=none
Test plan
- In a course with a user enrolled in 2 sections,
launch an LTI as that user
- Verify that you can see both section names in the
form data for the lti launch
Change-Id: I922b791a702949b0fe53de1f2118d8e6f01779d8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258586
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
closes OUT-4033
[fsc-timeout=50]
flag=reassign_assignments
This feature allows teachers to reassign an assignment
to a student after they've submitted to the assignment
and the teacher has provided feedback via comments.
The reassignment will appear in their planner with
a "Redo" tag visible next to the assignment.
Only supports assignments of submission type "Online".
test plan:
- enable "Reassign Assignments" feature option
- in a course, create a student account
- as a teacher, create an assignment with a due date
- as a student, confirm the assignment appears in
their planner
- as a student, submit to the assignment
- as a teacher, open the submission in SpeedGrader
- reassign the assignment to the student by:
* adding a comment to the assignment
* clicking the "Reassign Assignment" button
- as a student, confirm the assignment appears
with a "Redo" pill in the planner, along with
the comment from the teacher
- as a student, resubmit to the assignment and
confirm the assignment shows as completed again
in the planner without the "Redo" pill
- repeat the above steps, except with an assignment
that has "Anonymous Grading" enabled and it should
behave in the same way
- repeat the above steps but with more students and
no due date on the assignment, but with various
assignment overrides with due dates and some students
assigned to those overrides (using course sections)
and confirm the "Resassign Assignment" button only
appears for those students with assignment overrides
(and hence due dates), with the button being disabled
until the student both submits to the assignment and
the teacher has provided a comment.
Change-Id: Id745b50f3810378804e0728e544ebf6bff8f756a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/251663
QA-Review: Kai Bjorkman <kbjorkman@instructure.com>
Product-Review: Jody Sailor
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Adrian Packel <apackel@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
fixes OUT-4196
flag=none
Test plan
- Set up an attachment on a question bank
(account and course, maybe?) and set the
root_account_id on the attachment to 0
- Run the migration and ensure the root_account
gets set correctly for the assessment_question's
root account
Change-Id: I02f7cefc161368d4760135464da01c3ab2d6359d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257985
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Mysti Lilla <mysti@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
This is part 3 of changing the datatype from varchar to UUID of
lookup_id and resource_link_id from lti_resource_links.
We start to read from the new columns created lookup_uuid and
resource_link_uuid.
Adding a migration to remove the not-null constraint of lookup_id and
resource_link_id columns. As part 3.1 we'll stop writing into these
columns, we need to execute this postdeploy migration at this point.
refs INTEROP-6488
flag=none
test-plan:
* specs should pass;
* you should check if LTI is launching as expected, and if the custom
params was expanded as expected in all records that were created in the
part 1 and 2;
* you should be able to new persist custom params, for example you can
use the RCE editor placement;
* you can follow the test-plan:
* https://gerrit.instructure.com/c/canvas-lms/+/256029
* https://gerrit.instructure.com/c/canvas-lms/+/254453
[fsc-timeout=30]
Change-Id: I401f53a82f4dbef66c45932eb2eed8727488313d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258246
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
and spec to match, now that rails 6 is default
Change-Id: Ic44f3fa370da80e363bd3085e559b8dfc9a04837
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/258426
Reviewed-by: Robin Kuss <rkuss@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
This is part 2 of changing the datatype from varchar to UUID of
lookup_id and resource_link_id from lti_resource_links.
We're adding a data fixup to fill the new columns `lookup_uuid` and
`resource_link_uuid`. In case, the value of the related columns
`lookup_id` or `resource_link_id` is invalid we'll generate a new UUID
and set the related columns as the same value to keep consistency.
refs INTEROP-6488
flag=none
test-plan:
* Before you apply the new migrations you should create some resource
links records and keep sure that `lookup_uuid` and `resource_link_uuid`
fields has nil value and `lookup_id` and `resource_link_id` fields has
invalid value;
* You should be able to execute the migration and it should works fine
without raising exception;
* You should check for those records created before, if lookup_uuid has
the same value of resource_link_id and if resource_link_uuid has the
same value of resource_link_id;
Change-Id: I34ebd3368e70cd1166349981e30e9bc9beaa197c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257953
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
closes DEMO-116
flag=none
test plan:
- create an online Canvas assignment with peer reviews settings
- create another online assignment, and see peer reviews
settings are retained on assignment creation page
- creat a NQ quiz
- check the NQ assignment in rails console, and all peer reviews
settings are reset to false/0
Change-Id: I4189aa9c3f48ee212a280568ecc4b14b89a93b4c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257264
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jared Crystal <jcrystal@instructure.com>
QA-Review: Mark McDermott <mmcdermott@instructure.com>
Product-Review: Han Yan <hyan@instructure.com>
Add ability to fetch all sub-outcomes of a learning outcome group
Ordered by parent group title then outcome short description. The
Results should be paginated to accommodate for infinite scroll.
closes OUT-4130
flag=improved_outcomes_management
test plan
- PreReq: Account/Course with outcomes and groups
- Log into Canvas and navigate to graphiql
- Run the below query
- outcomes should be populated with all outcomes and suboutcomes
For the root outcome group order by group title, outcome short
description
- Run the below query
query GroupDetailQuery($id: ID!, $outcomesCursor: String) {
group: legacyNode(type: LearningOutcomeGroup, _id: $id) {
... on LearningOutcomeGroup {
_id
description
title
outcomesCount
outcomes(first: 10, after: $outcomesCursor) {
pageInfo {
hasNextPage
startCursor
endCursor
}
nodes {
... on LearningOutcome {
_id
description
title
}
}
}
}
}
}
{
"id": 1,
"outcomesCursor": ""
}
- All outcomes should be returned ordered by parent group name,
then outcome name
- Change the name of an outcome and/or group to force it to change
the order.
- Run the query again, and you should see the updated outcome or
Parent group’s outcomes first before the remain outcomes
Change-Id: Icdad82c0b57ca625924d9b0af7d7b3304b88c251
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257653
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
QA-Review: Pat Renner <prenner@instructure.com>
refs INTEROP-6483
flag=none
See the discussion: https://gerrit.instructure.com/c/canvas-lms/+/257523/
5/app/controllers/submissions_api_controller.rb#625
The current behavior of inferring content-type/extension only considers
name or filename or url parameter to infer the extension, but when a name
doesn't contain the file extension and filename/url contains it, we can
use these fields to infer the content type/extension.
So, we're changing this behavior to infer from filename/url when it is
possible. When the name/filename/url contains an unknown mime type like
`example.xpto`, we'll return the content_type as `unknown/unknown` and
the extension as `xpto`.
test-plan:
* All steps from https://gerrit.instructure.com/c/canvas-lms/+/257523
should work fine;
* You should configure the assignment to restrict upload file types
like: png,xpto (invalid/unknown mime type);
* You should be able to submit the assignment and validate the file
created via Submission Details page, after:
* Informing a name without an extension and filename/url with extension;
* Informing a name/filename/url with an unknown extension;
Change-Id: I173c5ca2a6f9832ab3da92fd858b65654139221f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257762
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
closes FOO-1552
flag = none
test plan:
• permissions around course_sections function as they did before,
but now they are granularized and permanent
Change-Id: I5918a4c6ccce2ccaaf87d4b92772c31ad3e6280c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257619
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
QA-Review: Charley Kline <ckline@instructure.com>
When we’re submitting a file using the LTI tool, in some situations,
we can face a tool that doesn't send the text field in the content item
payload. This field is used to infer the filename and extension of the
submission file.
As a fallback, we'll try to recover the filename from the URL.
closes INTEROP-6483
flag=none
test-plan
* Have an LTI tool installed (you can use lti-1.3-test-tool); You could
use this commit https://gerrit.instructure.com/c/lti-1.3-test-tool/+/257525
to facilitate the tests in case it was not submitted yet.
* Have a Course recorded;
* Have a Student enrolled in this Course;
* Have an Assignment with online submission recorded to this Course;
* As a Student, you should be able to access the Assignment to submit
your homework. On the submission homework page, you should be able to
find LTI tool (usually rendered in a tab);
* After launching the tool, you should be able to select File as the
Content Item Type and fill the File URL field and submit;
* After submitting the Assignment, you should be able to access the
Submission Details page, and validate if the specified file was properly
created/recorded (name and image);
* I used https://dummyimage.com to generate some images like:
* https://dummyimage.com/300/09f/sample
* https://dummyimage.com/300/09f/sample.png
* https://dummyimage.com/300/09f
* https://dummyimage.com/300
* You should check if when the LTI tool sends the File Name (text) field
and everything works as expected;
Change-Id: Iae5f128492834f8efc8934de347cfa5d9944ac1b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257523
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
closes FOO-1501
refs FOO-130
flag = granular_permissions_course_files
[fsc-max-nodes=18]
Test Plan:
• see test plan outlined in base commit: g/253777
Change-Id: I33984062fd236348d39262395e5f51335e327ed9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256914
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Note: we'll want to re-run the data fix-up when we're ready to turn
on the feature flag permanently; in hopes to capture any differences
made to course files permissions between now and then.
Modified the files_controller quota and api_quota permission checks
to make them more lenient in regards to accepting any or all of the
files permissions role overrides. This allows legacy grouping and
new granularized files permissions to live in harmony and be modified
without causing unauthorized errors on the quota resource.
This commit will cover the backend permissions required to granularize
files / folders permission calls, however there will be a follow-up
ps to clean up the course file page to hide elements the user might
not be authorized to use.
closes FOO-130
refs FOO-1501
flag = granular_permissions_course_files
[fsc-max-nodes=18]
[fsc-timeout=30]
Test Plan:
- Run the migration and make sure there are no errors
- With the granular_permissions_course_files FF turned off,
course sections and REST API should work the same with this patch
set checked out as it does in beta/production
- Some things to check:
* How it acts as a teacher, student, and public user
in course files/folders and personal files/folders
with the various settings above toggled to different states
* How it acts as a teacher, student, and public user
in discussions, modules, content migrations/import/exports
(RCE should behave similarly throughout the site)
- With the granular_permissions_course_files feature flag turned on
course files/folders and REST API should work as expected. The same
list checked above should be done so again, but this time:
* Should only be able to upload or add folders if the
Course Files - add permission is enabled for the user's role
* Should only be able to manage file access, usage rights, move,
or rename course files/folders if the Course Files -
edit permission is enabled for the user's role
• Check Toolbar header at the top of Course files
• Check Cog (hamburger menu) to the right of each file/folder
• Check Usage Rights Indicator under usage rights column
that can be found in course and group file pages. This can
be enabled under course settings if not available
* Should only be able to delete course files/folders if the
Course Files - delete permission is enabled for the user's role
* Any given user/role should have full access to their respective
personal files/folders regardless of granted permissions. The
same also applies to a group context with some caveats
• Should not be able to modify file access in a group context
• Should not be able to modify usage rights in personal files
* A student enrollment not granted any file permissions (the default)
should only be able to _view_ and _download_ files unless granted
additional access from an authorizing role
* REST API works as expected
* UI works as expected with no additional javascript errors
Change-Id: Ieb2d10915c274959e8da4c623f7aba11d3540c2b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253777
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Product-Review: August Thornton <august@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
closes INTEROP-6436
flag=none
Test plan:
- make a bad grade passback request and check that logs output the
expected base string.
Change-Id: I0db7cb0adeb5fbf1b0e91a77693953f470385656
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257228
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
QA-Review: Wagner Goncalves <wagner.goncalves@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
fixes FOO-1166
so that the query for sub-accounts is not synchronous to the request
this should be api-compatible with the old method of one progress
object per sub account
note that sub-account themes are no longer generated in parallel,
so this may be a little bit slower than before. if it's an issue,
additional work may be required
test plan:
* have an account with lots of sub accounts
* edit and apply a theme
* progress bar shoould show correctly
* the theme should be applied to all sub accounts
Change-Id: I8c2fb4f38e72783052eaa0440072b6c8bd25bea8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/257004
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Updates LearningOutcomeGroupType to return the number of
nested subgroups and nested outcomes for a learning outcome
group.
closes OUT-4022
flag=improved_outcomes_management
Test plan:
- Create nested learning outcome groups
- For each nested learning outcome group create learning outcomes
- Make a query in /graphiql for an accout or a course and
retrieve the parent learning outcome group. e.g.
```
query MyQuery {
account(id: 1) {
rootOutcomeGroup {
id
_id
title
childGroupsCount
outcomesCount
}
}
}
```
- Compare the values at the fields childGroupsCount and
outcomesCount with the amount of nested subgroups and nested
outcomes respectively so they should match
Change-Id: Ie63a5b134d661832f5a22714b693dddc6887cec3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254149
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
refs INTEROP-5992
flags=none
Test plan
- Launch an LTI 1.1 tool as a site admin ONLY
role
- Ensure the "roles" form_data shows up with
'SysAdmin' instead of 'None'
- (Obviously the string is way longer, but
you get the idea)
Change-Id: Ib7b1dbf0c870733f29597d4891a583ad15ab1da1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256353
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Reviewed-by: Wagner Goncalves <wagner.goncalves@instructure.com>
fixes VICE-1001
flag=deprecate_sms
The sms_allowed account setting was used to allow institutions to
opt out of the deprecation. As we've cleaned up all such institutions
and don't intend to support this behavior any more, the setting was
also removed.
test plan:
- make sure you have a mobile device set up as a communication channel
- this can be done on /profile/settings under 'ways to contact'
- enter whatever number you'd like
- in a rails console, you can find your confirmation code with
> CommunicationChannel.last.confirmation_code
- navigate to /profile/communication
- the SMS options should be limited to announcements and grading
- message delivery/prevention is adequately tested in unit tests and
difficult to manually test without AWS resources
qa risk: low
Change-Id: I59ac6f020ad57da39f05f37dd6cf754f86daed8d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256900
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Caleb Guanzon <cguanzon@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Caleb Guanzon <cguanzon@instructure.com>
when exporting rich text containing media objects, these
get written to media_objects/<media_id> in the export
package--unless the media object is linked to a file,
in which case we declined to write a second copy
the problem is, media links in the export were rewritten
to use the media_objects/<media_id> path anyway, and
these links were broken if the file was exported.
to fix this, we will always use the file's path in the
export if the media object is linked to a file
test plan:
- have canvas talking to a working notorious instance
- upload/record a video in a rich text (i.e., in a page)
- upload a video to files, and embed the video in a page
- ensure the videos play
- ensure the course can be copied and the videos play
(they will keep the same media id)
- ensure the course can be exported to a common cartridge
and re-imported to a new course, and the videos play
(they will have new media ids)
also:
- enable offline web exports in the account
- add the two pages to a module
- perform an offline web export from the modules page
- extract the package and open index.html in a browser
- the videos should play
(note that they look weird in safari; this is something
we should address separately)
fixes LS-1729
Change-Id: Ib913ee990886ea734f3a29893705a9d382e46e0c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256655
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Evan Battaglia
