test plan:
see coverage in previous rspec/selenium builds, same numbers as before
Change-Id: I331bf5102914da00a5d350f32b74b4bc9d49c5f8
Reviewed-on: https://gerrit.instructure.com/106895
Tested-by: Jenkins
Reviewed-by: Landon Wilkins <lwilkins@instructure.com>
Product-Review: Landon Wilkins <lwilkins@instructure.com>
QA-Review: Landon Wilkins <lwilkins@instructure.com>
This adds the ability to see the breakdown of grades for
essay/file-upload question types. They are broken up as the top 27%,
the bottom 27%, the middle 46%, and any ungraded answers, then displayed
in table format.
Refs CNVS-25737
Closes CNVS-26756
Test Plan:
- Check that essay+file_upload question type tables look good
- Buckets show themselves as "correct" if they contain a student
score >= to the question points possible
- Buckets can contain more students than 27% if the scores are
identical (All 100% scores show up in top bucket/middle bucket
even if 100% are 100% scores)
- New answer tables are accessible like other tables
- Buckets are explicitly ordered as top, middle, bottom, ungraded
Change-Id: I62798938b9176de97df2e498a2f9b3b02a81086c
Reviewed-on: https://gerrit.instructure.com/70907
Tested-by: Jenkins
Reviewed-by: John Corrigan <jcorrigan@instructure.com>
Reviewed-by: Davis McClellan <dmcclellan@instructure.com>
QA-Review: Michael Hargiss <mhargiss@instructure.com>
Product-Review: Jason Sparks <jsparks@instructure.com>
rely on test_all_gems.sh to output header and trailer,
and use `set -e` in each test.sh to simplify passing
through errors
Change-Id: I3ba724ad2539ddfe31195394c43f646acfc73920
Reviewed-on: https://gerrit.instructure.com/70469
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Adds support for the numerical precision answer types in Quiz
Statistics, thus preventing a page error on any quiz that has that
question type.
Closes CNVS-25821
Test Plan:
- Make a quiz with a numerical precision question type
- Confirm that quiz stats page renders
- Confirm that question details shows a reasonable representation of
the quiz question answer. i.e. "1 (with precision: 5)"
Change-Id: I2935a1aeb1c1c238184f8e607708b32189b0164b
Reviewed-on: https://gerrit.instructure.com/69095
Tested-by: Jenkins
Reviewed-by: Matt Berns <mberns@instructure.com>
Reviewed-by: Davis McClellan <dmcclellan@instructure.com>
QA-Review: Rajkumar Kethavath <rkethavath@instructure.com>
Product-Review: Jason Sparks <jsparks@instructure.com>
test case:
1. Create a Quiz with a question for each type.
2. Add at a few users to the course.
3. Create student submissions for the quiz with varying responses
4. When you log in as a teacher navigate to that quizzes statistics,
and click on the expansion button for a question.
5. Click on the response ratio for that question and a modal will show
displaying all the users that have responded to that question.
Change-Id: I5e23b42f6982683902124d734c36eedb9c5da7f7
Fixes: PFS-2227
Reviewed-on: https://gerrit.instructure.com/59720
Reviewed-by: Ryan Taylor <rtaylor@instructure.com>
Tested-by: Jenkins
QA-Review: Deepeeca Soundarrajan <dsoundarrajan@instructure.com>
Product-Review: Jason Sparks <jsparks@instructure.com>
also make it easier to find which gem failed specs
Change-Id: I4cfc6edcea1cf5d1c78043c4eda7db9998ed1f12
Reviewed-on: https://gerrit.instructure.com/56181
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
* add test runner for broadcast_policy and fix a couple of tests that
had been broken by accident
* add test runner for canvas_quiz_statistics and add a reference to
iconv to it's Gemfile to allow tests to run in ruby 2
* add a test runner for canvas_sanitize
* remove leftover cruft from running gem builds in both rails 2 and 3
Change-Id: Iab2a0986f277a82c096e1fff2dab1d3a55b91733
Reviewed-on: https://gerrit.instructure.com/54518
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Change-Id: Ib7db93b7ce5c877d2c0dcf24ee0d748f11ec58cb
Reviewed-on: https://gerrit.instructure.com/44806
Reviewed-by: Adam Ard <aard@instructure.com>
Product-Review: Adam Ard <aard@instructure.com>
QA-Review: Adam Ard <aard@instructure.com>
Tested-by: Adam Ard <aard@instructure.com>
Ports the generation of stats for that question type to the CQS gem.
Changes:
- no longer exposing "user_ids"
- can now identify students who skipped the question
Closes CNVS-13089
TEST PLAN
---- ----
- create a quiz with multiple-answer question(s)
- take it by a number of students and cover the following cases:
- answer correctly by picking only the right choices
- answer almost correctly by:
1. picking only 1 right choice
2. picking 1 right and 1 wrong choices
3. picking everything
- answer incorrectly by picking only the incorrect choice(s)
- don't answer at all
- get the stats from the API:
- for "responses", "correct", and "partially_correct", verify they
meet the specification in the docs
- also for the "responses" field in each document in "answers"
- verify that there is an answer document with "none" for an id with
"responses" that reflect the number of students that skipped the
question
- visit ember quiz stats:
- verify the "Attempts: X out of Y" should read the "responses"
field out of the total quiz participant count
- verify the donut chart reads the correct "correct" response ratio
- verify there is a "No-Answer" bar
- expand the question details:
- verify that all the choices are displayed, and the correct
choices are highlighted in GREEEN
Change-Id: Ibc08b6f521f9cae35dd16950c68c164d7e27d95d
Reviewed-on: https://gerrit.instructure.com/35736
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Closes CNVS-13386
TEST PLAN
---- ----
- similar to multiple-choice in https://gerrit.instructure.com/35734
but with this question type with a few minor differences:
- "correct" metric, see docs
- "other"/"unknown" answer should be generated only if any student
writes an answer that is not correct
Change-Id: I2d56afc0d9c1a3620d4de96059ebdadc56baecf9
Reviewed-on: https://gerrit.instructure.com/35735
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Changes:
- no longer exposing user_ids
- no longer exposing response_values
- new metric: "responses" at the root-level
Closes CNVS-13385
TEST PLAN
---- ----
- create a quiz with those question types
- take the quiz by a number of students, choose different answers
- get the stats from the API:
[GET] /api/v1/courses/:course_id/quizzes/:quiz_id/statistics
- verify you get the documented metrics and they have the proper
values
- verify the relevant sections in ember quiz stats is still functional
Change-Id: I470912060042405629812585ef6667693b1302a1
Reviewed-on: https://gerrit.instructure.com/35734
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Product-Review: Derek DeVries <ddevries@instructure.com>
Adds numerical question support to CanvasQuizStatistics and the renderer
in the Ember quiz statistics (subclass of ShortAnswer question).
Closes CNVS-13087, CNVS-13086
TEST PLAN
---- ----
- create a quiz with a numerical question:
- mix and match between the answer types: exact, exact with margin,
and range answers
- take the quiz by a number of students, we need to score like this to
cover the possible cases:
- a correct answer
- a missing answer
- an incorrect/out-of-range answer
- hitting the stats api should return the documented metrics in the
output of "question_statistics":
/api/v1/courses/:course_id/quizzes/:quiz_id/statistics [GET]
- visiting ember quizzes should now render the numerical stats which
is similar to short answer (fill in the blank) and multiple answers:
- make sure the "Other" and "No Answer" answers and bars show up
- verify the API docs are updated
Change-Id: I0c9ec0dbc7c729410241177b6ed43b0cd4dad143
Reviewed-on: https://gerrit.instructure.com/35431
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Product-Review: Derek DeVries <ddevries@instructure.com>
Adds support for Matching question statistics. Similar to FIMB and
Mulitple Dropdowns in the output.
Closes CNVS-13088
TEST PLAN
---- ----
Similar to https://gerrit.instructure.com/#/c/35101/ but with Matching
questions instead of FIMB.
Change-Id: Iaa5da1631503a8750b9673d7cb6064a296a8aa55
Reviewed-on: https://gerrit.instructure.com/35331
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Render question statistics for those question types similar to Essay.
File Upload should include a link to download all submissions too.
QuizSerializer was updated to include the submission ZIP download URL.
Closes CNVS-12988
TEST PLAN
---- ----
- using your data set from https://gerrit.instructure.com/#/c/35112/
- verify that the score chart renders with the student scores
- verify that you get a link to Download All Files for FUpload
questions
- verify that the "Attempts: x out of Y" reads correctly based on
the "responses" field and the total participant count,
respectively
- tooltips and chart interactivity like that for Essay
Change-Id: I4a77631491b169106e2eb677b21c1f30f3ca9440
Reviewed-on: https://gerrit.instructure.com/35113
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jason Madsen <jmadsen@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Add support for generating stats for File Upload and Formula questions.
Similar to Essay metrics but adjusted to calculate properly.
Closes CNVS-13169
TEST PLAN
---- ----
- create a quiz with those question types
- take the quiz:
- answer both question types by at least one student, but leave it
unanswered by another so we can test the "responses" metric
- test the stats:
GET /api/v1/courses/:course_id/quizzes/:quiz_id/statistics
- the "responses" metric should count the number of students who
provided an answer
- the "graded" metric should read the number of students whose
answers you've graded so far
- the "full_credit" one should read 0 until you grade it and give
them a score higher than, or equal to, the maximum points possible
- "point_distribution" is similar to that of Essays; it is an array
of objects that track every score you gave the students and the
number of students who received those scores
- grade the scores and re-test to verify the metrics update correctly
- verify that the API documentation is updated to include those
question types
PS: the "responses" field will read "the number of students who uploaded
a file" for File Upload, and "the number of students who wrote any
answer" for Formula.
Change-Id: I890eafe018e000eef88de782bd7e86b5259df5d5
Reviewed-on: https://gerrit.instructure.com/35112
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jason Madsen <jmadsen@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Closes CNVS-13161
TEST PLAN
---- ----
- create a quiz with multiple-dropdown questions
- take the quiz by a number of students and type in different answers
- make an API request to stats:
GET /api/v1/courses/:course_id/quizzes/:quiz_id/statistics
- verify you get the documented metrics and they have the proper
values
- visit the ember quizzes page and verify that:
- the charts are still working
Change-Id: I37ef48f22c3c16170034b6dd887b40ff20da4af5
Reviewed-on: https://gerrit.instructure.com/35104
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Support for FIMB question statistics in the CanvasQuizStatistics gem.
Closes CNVS-13160
TEST PLAN
---- ----
- create a quiz with FIMB questions
- take the quiz by a number of students and type in different answers
- make an API request to stats:
GET /api/v1/courses/:course_id/quizzes/:quiz_id/statistics
- verify you get the documented metrics and they have the proper
values
- visit the ember quizzes page and verify that:
- the charts are still working
Change-Id: I2d82c708614c41e222aa24d65de64555d8056a9a
Reviewed-on: https://gerrit.instructure.com/35101
Reviewed-by: Jason Madsen <jmadsen@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Closes CNVS-13159
TEST PLAN
---- ----
Only code changes. Test plan would be the same as
https://gerrit.instructure.com/#/c/35096/
Change-Id: I0f26a3df1278d555237c803cebbe80cde1559065
Reviewed-on: https://gerrit.instructure.com/35097
Reviewed-by: Jason Madsen <jmadsen@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Ensure all input hashes are using symbol keys.
Closes CNVS-13158
TEST PLAN
---- ----
Code changes. Test plan is similar to
https://gerrit.instructure.com/#/c/35096/
Change-Id: Ie1db45e2a5875e410a758c0b2f14594345212b5d
Reviewed-on: https://gerrit.instructure.com/35098
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
- removed the question analyzer
- each answer analyzer is now expected to calculate its own
"responses" field as opposed to doing it in a generic manner
- made the Essay analyzer generate its "responses" field
- a self-documenting DSL for defining analyzer output metrics
Closes CNVS-13157
TEST PLAN
---- ----
It's all code movement/refactor so nothing new to test. The plan is to
verify the Essay question stats are unaffected (which is basically all
the gem supports at this point:)
- create a quiz with an essay question
- take the quiz by a few students
- grade a submission
- retrieve the stats via the API:
GET /api/v1/courses/:course_id/quizzes/:quiz_id/statistics
- verify the essay stats in the API are still functional, the
"responses" field in particular
Change-Id: I42b9552c60ccb56f7c6912fed7cc1173da71852d
Reviewed-on: https://gerrit.instructure.com/35096
Reviewed-by: Jason Madsen <jmadsen@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Essay stats include a donut chart of the ratio of students who received
full-credit, as well as an area graph showing the score distribution.
Also, the serializer now prepares the data in a way for ED to properly
populate the questionStatistics association with quizStatistics, so we
got rid of the hack in the route's #afterModel hook, and exterminated
the bug that caused the question stats not to be updated when the quiz
changes (e.g, via the URL).
Closes CNVS-12466, CNVS-13009, CNVS-13021
TEST PLAN
---- ----
- create a quiz with essay questions
- take the quiz by a number of students
- before grading, take a look at the stats page, make sure nothing
funny happens
- grade the responses with different scores
- revisit the page, make sure the graphs are rendered appropriately
- clicking the View in SpeedGrader link should open a new tab with the
SpeedGrader for the quiz
- hover over the circles and verify that you get tooltips similar to
those for MChoice and T/F questions
- verify that sorting by DI/Position is again functioning correctly
Change-Id: I3ab6818d86632720c6da2c37b2126b4abe63cefa
Reviewed-on: https://gerrit.instructure.com/33991
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Jason Madsen <jmadsen@instructure.com>
Product-Review: Derek DeVries <ddevries@instructure.com>
Refactoring the generation of quiz question statistics into its own gem.
This patch adds support for Essay question statistics.
Closes CNVS-12725
TEST PLAN
---- ----
- create a quiz with an essay question
- perform an API request to retrieve the quiz statistics
- ensure that the following metrics are generated and correct:
- "graded": number of students whose answers have been graded by the
teacher
- "full_credit": number of students who received a full score
- "point_distribution": a list of scores and the number of students
who received them (so if 2 students got graded for 3 points, it
should have a key of 2 and a value of 3), un-graded submissions
should be keyed under null
- "responses": number of students who answered the question
(wrote anything)
- "user_ids": IDs of those students
- documentation for QuizStatistics -> Essay should be updated with the
new stats
> Other things to test
- verify that the old statistics page still renders:
/courses/:course_id/quizzes/:quiz_id/statistics
- verify that you can still generate both student and item analysis
CSV reports
API endpoint for quiz stats:
/api/v1/courses/:courseid/quizzes/:quiz_id/statistics [GET]
Change-Id: Ib15434ff4cef89ac211c1f4602d1ee609ef48ec4
Reviewed-on: https://gerrit.instructure.com/33990
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
Landon Wilkins