[skip-stages=Flakey]
Change-Id: I6abefdfa9fed6dd4525c8786e93efa548b3710f2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/319603
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Isaac Moore <isaac.moore@instructure.com>
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Build-Review: Jacob Burroughs <jburroughs@instructure.com>
Migration-Review: Jacob Burroughs <jburroughs@instructure.com>
This includes optimizations to conversion of OS results json
to list of LearningOutcomeResults
closes OUT-5430
flag=outcome_service_results_to_canvas
test plan:
- Jenkins pass
Change-Id: Ib76ef4f3adaf54e28a25e7467ce0c00b7354cf32
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/311261
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Dave Wenzlick <david.wenzlick@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Dave Wenzlick <david.wenzlick@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
This PS also corrects rubric filtering for the current course
closes OUT-5430
flag=outcome_service_results_to_canvas
test plan:
- Jenkins Passes
Change-Id: I3458eaefeff0832a01318e3446760c651f572375
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/309489
Reviewed-by: Martin Yosifov <martin.yosifov@instructure.com>
Reviewed-by: Dave Wenzlick <david.wenzlick@instructure.com>
QA-Review: Dave Wenzlick <david.wenzlick@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
closes OUT-5299
flag=outcome_service_results_to_canvas
test plan:
- tests pass in Jenkins: please pay extra attention
to the test plan. These tests should line up with
the tests in learning_outcome_result_spec.rb to
ensure all assignments that are classified as muted
are removed from the OS results.
- For testing in the UI:
- In a course with outcomes, create 2 New Quizzes
- Each quiz should be aligned to a different outcome
- Turn on LMGB, sLMGB, and Outcome Service Results to
Canvas FF on.
- Take each quiz as a student
- As a teacher, confirm:
- both outcomes have results in the LMGB
- both outcomes have results in the sLMGB
- results will be identified as a mastered/unmaster
pill. The list of assignments & its aligning
mastery in PS OUT-5297 & OUT-5298
- As a student, confirm:
- both outcomes has results in the sLMGB
- As a teacher, mute 1 quiz in the gradebook:
https://community.canvaslms.com/docs/DOC-12961-4152724339
- As a teacher, confirm:
- both outcomes are displaying in the LMGB & sLMGB
- As a student, confirm:
- Only 1 outcome results is displaying in the sLMGB
- As a teacher, mute the 2 quiz in the gradebook and
confirm:
- both outcomes are displaying in the LMGB & sLMGB
- As a student, confirm:
- 0 outcome results is displaying in the sLMGB
- As a teacher, unmute both quizzes & confirm:
- both outcomes has results in the LMGB & sLMGB
- As a student, confirm:
- both outcomes has results in the sLMGB
Change-Id: I21e085b3e856410cfe89ce57db2271f05858c097
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/302565
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Dave Wenzlick <david.wenzlick@instructure.com>
QA-Review: Dave Wenzlick <david.wenzlick@instructure.com>
Product-Review: Chrystal Langston <chrystal.langston@instructure.com>
closes OUT-5127
flag=outcome_service_results_to_canvas
- Test plan:
1. Turn on outcome_service_results_to_canvas FF
2. Turn on Learning Mastery Grade book FF
3. In a course, create the following:
1. Outcomes:
- Outcome 1, Outcome 2, Outcome 3
2. Course Item Bank
- Align the bank to Outcome 1
- Add one question to it
3. Classic Quizzes:
- Classic Quiz 1
- Add the question from the Course Item Bank
- Classic Quiz 2
- Add the question from the Course Item Bank
4. New Quizzes
- New Quiz 1
- Add 1 question and align the whole quiz to Outcome 2
- New Quiz 2
- Add 1 question and align the whole quiz to Outcome 3
5. 3 Students
- Student A, Student B, Student C
4. Load LMGB - results should be empty but should see Outcomes
associated to the Course
5. Act as Student A
- Take Classic quiz 1 and answer correctly
- Take Classic quiz 2 and answer incorrectly
6. Stop acting as Student A
7. Load LMGB
- Results should show for Student A
8. Act as Student B
- Take New Quiz 1 and answer correctly
9. Stop acting as Student B
10. Load LMGB
- Results should show for Student A & B
11. Act as Student C
- Take New Quiz 2 and answers correctly
12. Stop acting as Student C
13. Load LMGB
- Results should show for Student A, B & C
14. Set the Student A as inactive
course = Course.find(#{course id})
user = User.find(#{student user id})
StudentEnrollment.where(
course_id: course.id,
user_id: user.id).update(
workflow_state: "inactive")
15. Reload LMGB
- Results should only show for Student B & C
16. Set Student B’s enrollment as concluded
course = Course.find(#{course id})
user = User.find(#{student user id})
StudentEnrollment.where(
course_id: course.id,
user_id: user.id).update(
workflow_state: "completed")
17. Reload LMGB
- results should only show for Student C
Change-Id: I9f2b6b242d9c1c2b4bd238dd11205c875660daa0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/301450
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Dave Wenzlick <david.wenzlick@instructure.com>
QA-Review: Angela Gomba <angela.gomba@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
closes OUT-5253
flag=outcome_service_results_to_canvas
test plan:
- Tests pass in Jenkins
- Since this is merely calling OS to retrieve results
- there is nothing physical to test outside of loading
- the LMGB/sLMGB without error. Once the merge PS (i.e.
OUT-5127) is in place, we will be able to test if
results are showing or not in LMGB/sLMGB.
- Scenarios to test LMGB/sLMGB load without error
- In a course with a classic quiz & new quiz aligned to
outcomes, take both quizzes as a student.
- In the course's settings, ensure the following FF are
enabled: Student Learning Mastery Gradebook, Learning
Mastery Gradebook, Outcome Service Results to Canvas
Reporting
- Click on Grades
- If the gradebook is loaded by default, click on the drop
down list from the left hand side that click on Learning
Mastery Grade book.
- Assert that the LMGB loads without error
- Click on a student and click on Grades
- Assert that the sLMGB loads without error
- Conduct the same test again but with the Outcome Service
Results to Canvas Reporting FF disabled
Change-Id: I071974597b7f22df24b1921a42e2c0dff1c39f09
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/300279
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Dave Wenzlick <david.wenzlick@instructure.com>
QA-Review: Angela Gomba <angela.gomba@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Change-Id: Iad400936d7e53a5f92644f260c95bfb5bf9e972f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/293144
QA-Review: Jacob Burroughs <jburroughs@instructure.com>
Product-Review: Jacob Burroughs <jburroughs@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
Reviewed-by: Alex Slaughter <aslaughter@instructure.com>
Migration-Review: Alex Slaughter <aslaughter@instructure.com>
[skip-stages=Flakey]
the balance. mostly. Lint/UriEscapeUnescape is put in the pending
block because it's so touchy, and I didn't want to deal with it
right now
all manual
Change-Id: Ibeb81e013f56f160d51f7d237a9bcfe98daa1e53
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277569
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
refs OUT-4247
Test plan:
- ensure results from aligned rubrics and from quizzes
are visible in the learning mastery gradebook
Change-Id: Ie9731f2c702f91b3da2ce4d8ae43f7045ee5509e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/260613
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
closes CNVS-51409
refs OUT-4149
flag=none
Test plan:
- create course with two users with the same name
- create two outcomes in course with calculation
method decaying_average
- create two assignments with rubrics referencing those
two outcomes
- assess the students on those assignments (ensure that
you assess both students on one assignment, then
both on the other), with different scores on the two
assignments
- verify that the learning mastery gradebook shows the
correctly averaged scores for each student on each
outcome
Change-Id: I3bb8970b850f8dfa4e531f647fe33c89e57c75d4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/256184
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Pat Renner <prenner@instructure.com>
Product-Review: Jody Sailor
closes OUT-4119
flag=account_level_mastery_scales
test-plan:
before checking out this PS, verify the following:
- enable FF, if not already
- create a mastery scale at the account level
- create an account rubric with an account outcome
aligned to it and align it to an assignment in a course
- assess a student using the rubric
- create a course mastery scale
- validate that visiting the SLMGB for the student
uses the outcome mastery scale for points_possible
rather than the course scale
after checking out:
- verify that when visiting the SLMGB, the course scale is used
- verify that when visiting the LMGB, the course scale is used
- turn off the FF, and verify that traditional outcome ratings
are used in the LMGB/SLMGB
Change-Id: I9565121cfd8fbb380d4e0ae19791cf8cef4a6f14
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/254560
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Jody Sailor
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
- passes mastery scale info as params rather than
passing context for RollupScore to query
closes OUT-4087
flag=account_level_mastery_scales
Test plan:
- verify rollups work as intended in lmgb
- verify performance for a reasonable number of
students and outcomes
Change-Id: Idae8fb535933e4fa533409189ea98904d8429671
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/253117
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Pat Renner <prenner@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
closes OUT-4042
flag=account_level_mastery_scales
test-plan:
- Generate student result data for assignments
aligned to an account rubric:
> Generate account level mastery scales, if not already
created (values between 0-100 makes things easier)
> Create an account rubric
> Create a course, students, with an assignment using
account rubric
> Assess a student using the account rubric assignment
- Create a course level mastery scale for this course
- Repeat the above steps, but with a course rubric
> Using vastly different numbers (1000, 900, ..,
helps make the differences apparent)
- Enable the LMGB, SLMGB feature flag
- Visit the LMGB
- Verify that only course level mastery levels are used in
the gradebook, and results associated with the account level
rubric have been scaled appropriately
- Verify that the LMGB column tooltip chart accurately
shows the percents using course level scales
- Verify the filters on the right of the LMGB are using course
level scales
- Turn off the account_level_mastery_scales FF
- Verify that the LMGB uses account scales for the filter
and column charts
- Verify that the results are now scaled using the
account level scale
Change-Id: Ie95f3347a0f1bd326d50c4adf6c972c0cf528715
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/252322
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Jody Sailor
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
on Learning Mastery Gradebook outcomes report the sorting was
changed to sort by student name
fixes OUT-226
flag=none
test plan:
- create a course and add in 3 outcomes with default settings
- add in 4 students enrollments to the course
- create an assignment and attach a rubric with 3 outcomes
- go into speedgrader for the assignment and use the rubric on
every student and give them different assessments
- go to course settings and turn on Learning Mastery Gradebook
- go to Grades and select Learning Mastery
- click export on the right
- open the csv, the order of students should be by student name
Change-Id: I6dbf00d32eeaf7a9866a9475bf497dc0eed770ed
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/248571
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Jody Sailor
closes: OUT-3769
flag=account_level_mastery_scales
Test Plan: 1. Create a course in local Canvas with two students,
assignments and an outcome attached with rubrics.
2. Create scores for each assignment high scores for the first assignment
and low scores for the second assignment scores
3. Will call this endpoint locally /api/v1/courses/1/outcome_rollups
?rating_percents=true&per_page=20&include[]=outcomes&include[]=users
&include[]=outcome_paths&page=1&sort_by=student
expecting to see the highest scores recorded
4. Next turn on the FF listed above
5. Call this endpoint locally /api/v1/courses/1/outcome_rollups?rating_percents=true
&per_page=20&include[]=outcomes&include[]=users&include[]=outcome_paths
&page=1&sort_by=student
expecting to see the latest/low scores recorded
Change-Id: I03bc06044632513b576a35ac18a92381211938e0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/243686
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
closes OUT-2899
test plan:
- in a course, create an outcome
- create 5 assignments aligned to that outcome
- as a student, submit to all the assignments
- as a teacher, provide a rubric assessment to all submissions
- in a separate browser tab, confirm that 5 results show up
(so no skipped results and no repeating results), in result id
order, using two different pagination approaches (replace
"1" with the actual course id and prefix each call with
the domain being tested, like http://canvas.docker):
* 2 results per page
/api/v1/courses/1/outcome_results/?per_page=2&page=1
/api/v1/courses/1/outcome_results/?per_page=2&page=2
/api/v1/courses/1/outcome_results/?per_page=2&page=3
* 10 results per page
/api/v1/courses/1/outcome_results/?per_page=10
Change-Id: I963c6c89e395e2a9de9914152cbd92f375ce8b4d
Reviewed-on: https://gerrit.instructure.com/176507
Tested-by: Jenkins
Reviewed-by: Matthew Berns <mberns@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Neil Gupta <ngupta@instructure.com>
closes OUT-2550
test plan:
- create more than 20 student users in a course.
you can bulk create users by modifying the sample
file of 10 users under "users.csv" on
https://community.canvaslms.com/docs/DOC-12585-4214164118
and then importing them through an account's SIS Import
page (you may need to enable "SIS imports" on the account
features section on its settings page).
afterwards, add the students to a course
- create a course outcome
- create an assignment, aligned to the outcome using a rubric
- as all the students, submit to the assignment
- as a teacher, in SpeedGrader, create rubric assessments for
all submissions, selecting different criterion ratings
- load the LMGB page
- hover over the outcome header, and confirm the outcome details
summarize the correct percentages based on the outcome scores
- navigate to the 2nd page of results
- hover over the outcome header, and confirm the outcomes details,
like described two steps above
- enable the New Gradebook
- repeat the steps above, starting at loading the LMGB page
Change-Id: I52815eb2e05a04a0ad70c9c53e13726dbc626b64
Reviewed-on: https://gerrit.instructure.com/173291
Tested-by: Jenkins
Reviewed-by: Neil Gupta <ngupta@instructure.com>
Reviewed-by: Frank Murphy III <fmurphy@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Neil Gupta <ngupta@instructure.com>
closes OUT-1859
test plan:
- in a course, create about 10 outcomes and 10 student users
- create an assignment, aligned to an outcome
- submit to the assignment with two student users
- assess the rubrics for each submission in speed grader
- load LMGB, confirm that all students and outcomes appear,
with the results of the two students
- using a tool like Postman, perform an API call with an
access to the following endpoint:
http://canvas.docker/api/v1/courses/<course id>/outcome_rollups
- confirm that a rollup appears for each student, and only scores
appear for students whose assignments were assessed
- append "?exclude[]=missing_user_rollups" to the request above
and resubmit
- confirm that only rollups with scores appears (should be two)
- append "&include[]=outcomes" to the request above and resubmit
- confirm that under the "linked" section, that "outcomes" contains
all outcomes in the course
Change-Id: I399715093943455659ed5a05c39befca8dc2e801
Reviewed-on: https://gerrit.instructure.com/156247
QA-Review: Dariusz Dzien <ddzien@instructure.com>
Tested-by: Jenkins
Reviewed-by: Neil Gupta <ngupta@instructure.com>
Reviewed-by: Frank Murphy <fmurphy@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
closes OUT-2304
performance:
Indices are used throughout the scoped query.
Shard.current.id => 1773
base query:
LearningOutcomeResult.active.where(context_code:'course_1079845',user_id:3306819,learning_outcome_id:1397026)
without scope:
----------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=4.86..8.20 rows=1 width=268)
-> Bitmap Heap Scan on learning_outcome_results (cost=4.42..5.54 rows=1 width=268)
Recheck Cond: ((user_id = 3306819) AND (learning_outcome_id = 1397026))
Filter: ((context_code)::text = 'course_1079845'::text)
-> BitmapAnd (cost=4.42..4.42 rows=1 width=0)
-> Bitmap Index Scan on index_learning_outcome_results_association (cost=0.00..1.73 rows=27 width=0)
Index Cond: (user_id = 3306819)
-> Bitmap Index Scan on index_learning_outcome_results_on_learning_outcome_id (cost=0.00..2.44 rows=123 width=0)
Index Cond: (learning_outcome_id = 1397026)
-> Index Scan using content_tags_pkey on content_tags (cost=0.43..2.66 rows=1 width=8)
Index Cond: (id = learning_outcome_results.content_tag_id)
Filter: ((workflow_state)::text <> 'deleted'::text)
with scope (`exclude_muted_associations`):
----------------------------------------------------------------------------------------------------------------------------------------------------------
Nested Loop Left Join (cost=6.99..20.54 rows=1 width=268)
Join Filter: ((learning_outcome_results.association_type)::text = 'Assignment'::text)
Filter: (((ra.muted IS NULL) AND (qa.muted IS NULL) AND (sa.muted IS NULL)) OR (ra.muted IS FALSE) OR (qa.muted IS FALSE) OR (sa.muted IS FALSE))
-> Nested Loop Left Join (cost=6.56..17.88 rows=1 width=270)
-> Nested Loop Left Join (cost=6.13..15.72 rows=1 width=277)
Join Filter: ((learning_outcome_results.association_type)::text = 'Quizzes::Quiz'::text)
-> Nested Loop Left Join (cost=5.71..13.06 rows=1 width=269)
Join Filter: (((rassoc.association_type)::text = 'Assignment'::text) AND ((rassoc.purpose)::text = 'grading'::text))
-> Nested Loop Left Join (cost=5.28..10.85 rows=1 width=293)
Join Filter: ((learning_outcome_results.association_type)::text = 'RubricAssociation'::text)
-> Nested Loop (cost=4.86..8.20 rows=1 width=268)
-> Bitmap Heap Scan on learning_outcome_results (cost=4.42..5.54 rows=1 width=268)
Recheck Cond: ((user_id = 3306819) AND (learning_outcome_id = 1397026))
Filter: ((context_code)::text = 'course_1079845'::text)
-> BitmapAnd (cost=4.42..4.42 rows=1 width=0)
-> Bitmap Index Scan on index_learning_outcome_results_association (cost=0.00..1.73 rows=27 width=0)
Index Cond: (user_id = 3306819)
-> Bitmap Index Scan on index_learning_outcome_results_on_learning_outcome_id (cost=0.00..2.44 rows=123 width=0)
Index Cond: (learning_outcome_id = 1397026)
-> Index Scan using content_tags_pkey on content_tags (cost=0.43..2.66 rows=1 width=8)
Index Cond: (id = learning_outcome_results.content_tag_id)
Filter: ((workflow_state)::text <> 'deleted'::text)
-> Index Scan using rubric_associations_pkey on rubric_associations rassoc (cost=0.42..2.64 rows=1 width=33)
Index Cond: (id = learning_outcome_results.association_id)
-> Index Scan using assignments_pkey on assignments ra (cost=0.43..2.19 rows=1 width=9)
Index Cond: (id = rassoc.association_id)
-> Index Scan using quizzes_pkey on quizzes (cost=0.43..2.65 rows=1 width=16)
Index Cond: (id = learning_outcome_results.association_id)
-> Index Scan using assignments_pkey on assignments qa (cost=0.43..2.15 rows=1 width=9)
Index Cond: (id = quizzes.assignment_id)
-> Index Scan using assignments_pkey on assignments sa (cost=0.43..2.65 rows=1 width=9)
Index Cond: (id = learning_outcome_results.association_id)
test plan:
- create two course-level outcomes
- create an assignment with a single question, and align the 1st outcome via a rubric
- create a quiz bank with a single auto-gradeable question (e.g. true/false), and
align the 2nd outcome to it
- create a quiz that pulls the single question from the quiz bank above
- as a student, submit to the assignment and the quiz
- as a teacher, assess the assignment, providing a score to the rubric
- as a teacher, confirm:
* both outcomes have results in the LMGB
* both outcomes have results in the sLMGB
- as a student, confirm:
* both outcomes have results in the sLMGB
- as a teacher, mute the assignment in the gradebook:
https://community.canvaslms.com/docs/DOC-12961-4152724339
- as a teacher, confirm:
* both outcomes have results in the LMGB
* both outcomes have results in the sLMGB
- as a student, confirm:
* only the outcome associated with the quiz bank has results in the sLMGB
- as a teacher, mute the quiz in the gradebook
- as a teacher, confirm:
* both outcomes have results in the LMGB
* both outcomes have results in the sLMGB
- as a student, confirm:
* no outcomes should have results in the sLMGB
- as a teacher, unmute the assignment in the gradebook
- as a teacher, confirm:
* both outcomes have results in the LMGB
* both outcomes have results in the sLMGB
- as a student, confirm:
* only the outcome associated with the assignment has results in the sLMGB
Change-Id: I0ea05eedd29383501cc9306bcedcfa67aee4cd67
Reviewed-on: https://gerrit.instructure.com/155210
Tested-by: Jenkins
Reviewed-by: Neil Gupta <ngupta@instructure.com>
Product-Review: Neil Gupta <ngupta@instructure.com>
QA-Review: Neil Gupta <ngupta@instructure.com>
closes OUT-2144
test plan:
- create a course-level outcome with default rubric criterion
- create an assignment that is aligned to that outcome
- create two sections in a course (in Settings)
- create 6 student users, split evenly between the two sections
- masquerade as each student and submit to the assignment
- in speedgrader, assess the rubrics with the following scores:
1st section students: 5, 3, 0
2nd section students: 4, 2, 1
- perform an authenticated API call using a tool like Postman
to fetch the average aggregate score:
> /api/v1/courses/<course id>/outcome_rollups?aggregate=course
- confirm the average score is 2.5
- perform the same call again, but fetch the median aggregate score:
> /api/v1/courses/<course id>/outcome_rollups?aggregate=course&aggregate_stat=median
- confirm the median score is 2.5
- determine the "section_id" values for each of the two sections above
by running this in a Rails console:
> CourseSection.last(2).map(&:id)
- append "§ion=<section id>" to the average aggregate score call
above, and confirm the average scores:
1st section: 2.67
2nd section: 2.33
- append ""§ion=<section id>" to the median aggregate score call
above, and confirm the median scores:
1st section: 3
2nd section: 2
Change-Id: I5701fd3edc1ff423caf4735406ee1bc3b5b1b011
Reviewed-on: https://gerrit.instructure.com/156486
Reviewed-by: Frank Murphy <fmurphy@instructure.com>
Tested-by: Jenkins
Reviewed-by: Matt Berns <mberns@instructure.com>
QA-Review: Dariusz Dzien <ddzien@instructure.com>
Product-Review: Augusto Callejas <acallejas@instructure.com>
closes OUT-1846
Test plan:
- create an outcome
- create two assignments with rubrics including
the outcome
- set one of the rubrics to "Don't post results to LMGB"
- assess a student using both rubrics with different scores
- visit the LMGB
- verify that only the non-hidden score is reflected
in the LMGB
- visit the student LMGB
- verify that only the non-hidden alignment is shown
in the student lmgb
- visit
http://canvas.docker/api/v1/courses/COURSE_ID/outcome_results?user_ids[]=USER_ID&outcome_ids[]=OUTCOME_ID
(replacing COURSE_ID, USER_ID and OUTCOME_ID)
- verify that hidden alignment is not shown
- add
&include_hidden=true
to the above URL
- verify that hidden alignment is now shown
Change-Id: Ie08c63f32686ffc42394e926c22eac773799d616
Reviewed-on: https://gerrit.instructure.com/154464
Tested-by: Jenkins
Reviewed-by: Frank Murphy <fmurphy@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
QA-Review: Augusto Callejas <acallejas@instructure.com>
Product-Review: Sidharth Oberoi <soberoi@instructure.com>
closes OUT-2300
test plan (dev-qa):
- create a new outcome
- create a new rubric and align the outcome
- align the rubric to a new assignment, ensure you
choose the box to remove points from rubric
- assess a student
- go to the LMGB
- the outcome_rollups response should include a
hide_points field in the json
- hide_points should be 'true' on the rollup for the new outcome
with the newly created outcome
- attach the newly create outcome to a different assignment, one
that doesn't have its points hidden on the rubric association
- assess the student
- return to lmgb
- hide_points should be 'false' for the rollup on the new outcome
Change-Id: I0f9accb4ff0cc76f40c08ce08e21091e35df6af8
Reviewed-on: https://gerrit.instructure.com/153547
Tested-by: Jenkins
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Product-Review: Michael Brewer-Davis <mbd@instructure.com>
QA-Review: Michael Brewer-Davis <mbd@instructure.com>
fixes OUT-460
test plan:
- create 2 outcomes, one with decaying average and one w/ n_mastery
- attach each outcome to an assignment and a quiz
- it's not reccomended to use exactly 5 questions for quiz testing
since this has the potential to obfuscate possible calc errors
- log in as a student and submit to the assignment/take the quiz
- as the teacher/admin, asses the outcome on the assignment in
speedgrader. It's reccomended to get a high score on at least
one quiz that's being tested, in order to ensure the mastery
score on the result does not exceed the max possible score
for the outcome
- view the students outcome scores in the lmgb and student lmgb
to confirm the score's accuracy
try various scores, but here's an initial example assuming
Outcome A is decaying avg, and Outcome B is n_mastery
Outcome A
- Attach Outcome A to two assignments and two quizzes
- Submit as the student, first to the two assignments,
scoring a 3.0 and a 2.0, then on a quiz in which you
get 90% on the aligned bank
- on the final/most recent bank, score a 40%
- the score for the Outcome should be 2.41
Outcome B
- Attach Outcome B to two assignments and three quizzes
- Submit as the student. Order does not matter, but ensure
scores of 3.0 and 3.5 on the assignments, and 20%, 50%,
and 80% on the quizzes.
- the score for the Outcome should be 3.5
Change-Id: If99d8ab6a3791137e407ab43fd8af2c0d69058d5
Reviewed-on: https://gerrit.instructure.com/93333
Reviewed-by: Augusto Callejas <acallejas@instructure.com>
Reviewed-by: Michael Brewer-Davis <mbd@instructure.com>
Tested-by: Jenkins
QA-Review: Cemal Aktas <caktas@instructure.com>
Product-Review: McCall Smith <mcsmith@instructure.com>
closes #CNVS-26031
Change-Id: I2e0351fb62e5a06b47fe8c6c3dd503318d29a7ad
Reviewed-on: https://gerrit.instructure.com/69228
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Closes CNVS-13587
Test Plan:
- Create an outcome.
- Enable the Learning Mastery Gradebook (LMG)
- Create an assignment
- Add a rubric to that assignment
- Add an outcome to that Rubric
- Submit that assignment as a student
- Give a student a point value on that outcome.
- Verify their information now shows in LMG
- Remove the Outcome from the Rubric
- Check LMG, outcome info should be gone
- Delete the rubric
- Check LMG, outcome info should be gone
- QA: be sure to check outcomes that use averaging as part of their mastery
evaluation (n_mastery and decaying average), just to make sure the math
still works as expected.
- Confirm the same behaviors when attaching/removing an outcome from a
quiz. (Bonus, unreported bug, I think).
Change-Id: Icdcf674a669165200150f84eb3d943863e09a49d
Reviewed-on: https://gerrit.instructure.com/62055
Reviewed-by: Brian Finney <bfinney@instructure.com>
Tested-by: Jenkins
QA-Review: Michael Hargiss <mhargiss@instructure.com>
Product-Review: Allison Weiss <allison@instructure.com>
refs CNVS-12716
test plan:
- make course with many assessed outcomes
- load the mastery gradebook
- it should load relatively fast
Change-Id: I81a91d58d37e5ae3d5399d59f6e3626af53496ea
Reviewed-on: https://gerrit.instructure.com/53493
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Deepeeca Soundarrajan <dsoundarrajan@instructure.com>
Tested-by: Jenkins
Product-Review: Hilary Scharton <hilary@instructure.com>
fixes CNVS-19658
test plan:
- go to a course with at least one student and assignment
- create a rubric with at least 2 criterion and attach it to the assignment
- pull up the assignment in speedgrader
- enter points for the first criteria and save
- reopen the rubric, points for the first criteria should persist, the second
criteria score should still be empty(it should not show '0')
- refresh the page, the state of criteria scores should persist
- change the score of the first criteria to 0.
- repeat prior two steps, the score of 0 for the first criteria should persist
- remove score entirely for first criteria
- repeat the re-open rubric and refresh page steps, the field for the first
criteria being empty should persist, it should not change to '0'
- change the score for a criteria to something with letters and save
- re-open the rubric, the score field should empty (repeat prior reopen and
refresh steps)
Change-Id: I20fca1894598269b1f34c2090d36a6bb4ff586dd
Reviewed-on: https://gerrit.instructure.com/52948
Product-Review: Hilary Scharton <hilary@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Matt Berns <mberns@instructure.com>
Tested-by: Jenkins
fixes CNVS-11656
test plan:
- enable the teacher and student learning mastery gradebook feature flags
- go to gradebook and select the Learning Mastery tab
- outcome scores per student and aggregate totals should appear the same as
they do on master
- hover mouse over outcome title to trigger popover
- popover should include a path to the outcome, as well as a pie graph with a
key to the right of it that shows colors for 'exceeds expectations',
'meets expectation', and 'does not meet expectations'
- popver should then show mastery points needed, outcome description,
then a content box that includes the calculation method, along with
method description/example.
- click a student name to go to student learning mastery gradebook
- all things should appear the same was as they did on
https://gerrit.instructure.com/#/c/45777/ excluding a minor change to
the wording for most recent assessment
- masquerade as a student and go to /grades in a course
- all popover content should still be the same as when viewing this page as a
teacher
Change-Id: Ibe4214a4217c4fb42824e66934b49dccff13fe17
Reviewed-on: https://gerrit.instructure.com/48004
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Adam Stone <astone@instructure.com>
Product-Review: Hilary Scharton <hilary@instructure.com>
Tested-by: Jenkins
fixes CNVS-12924
test plan:
- enable student learning mastery gradebook and go to its page
- the arrow on the left hand side of outcome groups should properly toggle when
expanding and collapsing
- the pill on the right hand side should properly show the amount of outcomes
attached to it(grey), and those that the student has mastered(green)
- expand a set of outcomes and hover over the icon on the right side
- a hover should appear showing specific details for the outcome including:
- a check mark, yellow plus, or red x for mastery, near-mastery, and remedial
- the last assignment and the date it was submitted (note: you will need to submit
and grade a new assignment to see this info. Outcomes prior to this update will
not have a submission time available and should display "N/A")
- what the current mastery evaluation method is. Methods should show as:
"Latest Score", "Highest Score", "Acheive Mastery (x) number of times", or
"(split) Decaying Average".
note: (x) should indicate number of times, (split) should indicate split
(e.g. 75/25, 65/35)
- Decaying Mastery and x number of times methods should also provide additional
example text of how the evaluation method works
- hover should disappear and reappear appropriate on mouse enter and leave
- check other locations where pills appear to ensure no unexpected
formatting changes
accessibility test plan:
- use keyboard navigation to get to the icon that creates the hover
- icon should prompt user to press space for more info
- the hover should expand and automatically set the focus to the content on the
top portion of the hover
- all fields should properly read out their information
- pressing escape should collapse the hover and return focus to the icon
Change-Id: Iaedc67794d2aceb681f5c2895684797f45be8c62
Reviewed-on: https://gerrit.instructure.com/45777
Tested-by: Jenkins
Reviewed-by: John Corrigan <jcorrigan@instructure.com>
QA-Review: Adam Stone <astone@instructure.com>
Product-Review: Hilary Scharton <hilary@instructure.com>
fixes CNVS-16915
(all mentions of 'gradebook' refer to learning mastery gradebook)
test plan:
- go to gradebook
- all existing mastery scores should still default to highest score to retain
accuracy of existing calculations
- in rails console, set 'calculation_method' to 'latest' for an outcome
- all results for that outcome should now show student's latest score on gradebook
- in rails console, set method to 'highest'
- all results for that outcome should now show student's highest score on gradebook
- in rails console, set method to 'n_mastery' and set 'calculation_int' to a number
between two and 5
- all results for that outcome should now show avg of student's higest n# of scores
(#n being the number set for the outcome's 'calculation_int')
- in rails console, set method to 'decaying_average' and set 'calculation_int' to 75
- scores should now appear as they did first time gradebook was viewed on step #2
- in rails console, set 'calculation_int' to 65
- scores for that outcome should now reflect decaying average with 65% weight given to
most recent score, and 35% weight for avg of all prior scores
- have a student make a new submission to an old assignment and give it a score
- the re-submission should now be most recent score for 'latest' and
'decaying_average' methods
- amend a score for an old submission without a new submission from the student
- the amended score should not become the most recent, but still be accurately
reflected in the 25%/35% part of the 'decaying_average' method
Change-Id: I330c47cb2288d99e589deca00ddb5a8ee8103a91
Reviewed-on: https://gerrit.instructure.com/44935
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Sean Lewis <slewis@instructure.com>
Product-Review: Benjamin Porter <bporter@instructure.com>
fixes CNVS-13365
test plan:
* open /api/v1/courses/1/outcome_rollups
* verify that each score includes a count
* verify that counts match the number of results for that
combination of outcome and student
Change-Id: I0cbbab7d2c1ebb6574e090046c52b7a4159bb9b9
Reviewed-on: https://gerrit.instructure.com/35708
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Trevor deHaan <tdehaan@instructure.com>
Reviewed-by: Drew Bowman <dbowman@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
also replaces raw hashes in outcome result analytics with
structs to improve error detection and possibly support active
model serialization in the future.
fixes CNVS-10037
test plan:
- check the outcome results api documentation for the new
aggregate parameter.
- call the outcome rollup api with the aggregate=course
parameter.
- verify the result in this case is one rollup that contains
an accurate average of the user rollups for each outcome.
Change-Id: I635cf5e28f5de73e44b63ac6234971657728d035
Reviewed-on: https://gerrit.instructure.com/28103
Reviewed-by: Joel Hough <joel@instructure.com>
Product-Review: Joel Hough <joel@instructure.com>
QA-Review: Joel Hough <joel@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
fixes CNVS-10198, CNVS-10201
test plan
- test outcome result api as in c/27631
- ensure that jsonapi paging metadata is returned
- fetch outcome results for a course that includes students
without outcome results (i.e. haven't submitted anything)
- ensure that the student is listed in the returned results
with an empty scores array
Change-Id: I00d8e9de241a243fb6ac1aa9f55150b8955a2452
Reviewed-on: https://gerrit.instructure.com/28015
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
QA-Review: Steven Shepherd <sshepherd@instructure.com>
Product-Review: Zach Pendleton <zachp@instructure.com>
Product-Review: Joel Hough <joel@instructure.com>
fixes CNVS-10034
test plan:
- check the new outcome results api documentation.
- generate some outcome results:
- create assignments with rubrics that have linked outcomes.
- grade those assignments with the rubric.
- call the /api/v1/outcome_results api endpoint.
- make sure the returned student scores contain the maxiumum
score achieved for each outcome.
- students should not be able to access the api.
Change-Id: Icddec2aa9000c01f3b05c1c3c21260f7c15f7f7e
Reviewed-on: https://gerrit.instructure.com/27631
Reviewed-by: Zach Pendleton <zachp@instructure.com>
Product-Review: Zach Pendleton <zachp@instructure.com>
QA-Review: Zach Pendleton <zachp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Jacob Burroughs