Quiz Stats [Backend] - Gem & Essays

Refactoring the generation of quiz question statistics into its own gem.
This patch adds support for Essay question statistics.

Closes CNVS-12725

TEST PLAN
---- ----

  - create a quiz with an essay question
  - perform an API request to retrieve the quiz statistics
  - ensure that the following metrics are generated and correct:
    - "graded": number of students whose answers have been graded by the
      teacher
    - "full_credit": number of students who received a full score
    - "point_distribution": a list of scores and the number of students
      who received them (so if 2 students got graded for 3 points, it
      should have a key of 2 and a value of 3), un-graded submissions
      should be keyed under null
    - "responses": number of students who answered the question
      (wrote anything)
    - "user_ids": IDs of those students
  - documentation for QuizStatistics -> Essay should be updated with the
    new stats

> Other things to test

  - verify that the old statistics page still renders:
    /courses/:course_id/quizzes/:quiz_id/statistics
  - verify that you can still generate both student and item analysis
    CSV reports

API endpoint for quiz stats:

  /api/v1/courses/:courseid/quizzes/:quiz_id/statistics [GET]

Change-Id: Ib15434ff4cef89ac211c1f4602d1ee609ef48ec4
Reviewed-on: https://gerrit.instructure.com/33990
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Caleb Guanzon <cguanzon@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>

.gitignore

@@ -54,3 +54,6 @@ Gemfile.lock3
 #remove this once we move jqeury into bower
 /public/javascripts/bower/jquery/
 
+[canvas-gems]
+/gems/*/coverage
+/gems/*/tmp

Gemfile.d/other_stuff.rb

@@ -125,6 +125,7 @@ gem 'canvas_ext', :path => 'gems/canvas_ext'
 gem 'canvas_http', :path => 'gems/canvas_http'
 gem 'canvas_kaltura', :path => 'gems/canvas_kaltura'
 gem 'canvas_mimetype_fu', :path => 'gems/canvas_mimetype_fu'
+gem 'canvas_quiz_statistics', :path => 'gems/canvas_quiz_statistics'
 gem 'canvas_sanitize', :path => 'gems/canvas_sanitize'
 gem 'canvas_sort', :path => 'gems/canvas_sort'
 gem 'canvas_statsd', :path => 'gems/canvas_statsd'

app/models/quizzes/quiz_question/essay_question.rb

@@ -26,17 +26,4 @@ class Quizzes::QuizQuestion::EssayQuestion < Quizzes::QuizQuestion::Base
     user_answer.answer_details[:text] = Sanitize.clean(user_answer.answer_text, config) || ""
     nil
   end
-
-  def stats(responses)
-    stats = {:essay_responses => []}
-
-    responses.each do |response|
-      stats[:essay_responses] << {
-        :user_id => response[:user_id],
-        :text => response[:text].to_s.strip
-      }
-    end
-
-    @question_data.merge stats
-  end
 end

app/models/quizzes/quiz_statistics/student_analysis.rb

@@ -357,6 +357,12 @@ class Quizzes::QuizStatistics::StudentAnalysis < Quizzes::QuizStatistics::Report
   #   "user_ids"=>[1,2,3],
   #   "multiple_responses"=>false}],
   def stats_for_question(question, responses)
+    if [ 'essay_question' ].include?(question[:question_type].to_s)
+      analyzer = CanvasQuizStatistics::QuestionAnalyzer.new(question)
+      analysis = analyzer.run(responses)
+      return question.to_hash.merge(analysis).with_indifferent_access
+    end
+
     question[:responses] = 0
     question[:response_values] = []
     question[:unexpected_response_values] = []

app/views/quizzes/quizzes/_question_statistic.html.erb

@@ -149,6 +149,7 @@
       </td>
     </tr>
   <% end %>
+  <% question[:unexpected_response_values] ||= [] %>
   <% if !question[:unexpected_response_values].empty? && (question[:question_type] == 'short_answer_question' || question[:question_type] == 'numerical_question') %>
     <tr class=" <%= 'group_row' if in_group %>">
       <td colspan="2">&nbsp;</td>

doc/examples/question_specific_statistics.md

@@ -123,8 +123,20 @@ may include extra metrics. You can find these metrics below.
 
 ```javascript
 {
-  // TODO
-  "essay_responses": null
+   // The number of students whose responses were graded by the teacher so
+   // far.
+   "graded": 5,
+
+   // The number of students who got graded with a full score.
+   "full_credit": 4,
+
+   // A set of maps of scores and the number of students who received
+   // each score.
+   "point_distribution": [
+     { "score": 0, "count": 1 },
+     { "score": 1, "count": 1 },
+     { "score": 3, "count": 3 }
+   ]
 }
 ```
 

gems/canvas_quiz_statistics/.rspec

@@ -0,0 +1,3 @@
+--color
+--format documentation
+--require spec_helper

gems/canvas_quiz_statistics/CHANGELOG.md

@@ -0,0 +1,12 @@
+## Changelog
+
+**05/11/2014**
+
+    - added support for counting the number of students who have provided an answer to the question being analyzer, the output is returned in the fields "responses" and "user_ids"
+
+**04/29/2014**
+
+  - added support for Essay question statistics with the following metrics:
+    1. "graded"
+    2. "full_credit"
+    3. "point_distribution"

gems/canvas_quiz_statistics/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+gem 'simplecov', '0.8.2', :require => false
+gem 'simplecov-rcov', '0.2.3', :require => false
+
+gemspec

gems/canvas_quiz_statistics/Guardfile

@@ -0,0 +1,9 @@
+# A sample Guardfile
+# More info at https://github.com/guard/guard#readme
+
+guard :rspec do
+  watch(%r{^spec/.+_spec\.rb$})
+  watch(%r{^lib/(.+)\.rb$})     { |m| "spec" }
+  watch('spec/spec_helper.rb')  { "spec" }
+end
+

gems/canvas_quiz_statistics/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2014 Instructure
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

gems/canvas_quiz_statistics/README.md

@@ -0,0 +1,30 @@
+# CanvasQuizStatistics
+
+A bunch of objects that can generate statistics from a set of responses to a quiz.
+
+_Work In Progress._
+
+## Extending
+
+### Adding support for a new question type
+
+**Implementing the analyzer**
+
+  - define an answer analyzer in `answer_analyzers/question_type.rb`
+  - make sure your analyzer implements the common interface, which is defined
+    by the `AnswerAnalyzer::Base` class
+  - please document both output *and* input formats that you expect to generate the stats
+
+**Registering it**
+
+Edit `lib/canvas_quiz_statistics/answer_analyzers.rb` and:
+
+  + require your analyzer
+  + add it to the list of available analyzers in
+    `CanvasQuizStatistics::AnswerAnalyzers::AVAILABLE_ANALYZERS`
+    where the key should be the question type (with the `_question` suffix)
+    and the value would be your analyzer
+
+**Covering it**
+
+You will probably need to simulate question data to cover your analyzer. Grab a JSON snapshot of the `question_data` construct for your question and save it in `spec/support/fixtures/` and check out the fixture helpers in `spec/support/question_helpers.rb` for more information on how to use the fixture.

gems/canvas_quiz_statistics/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

gems/canvas_quiz_statistics/TODO.md

@@ -0,0 +1,14 @@
+## TODO
+
+  - figure out whether an answer is present in a response to a question (should be overridable by each question type if needed) _[DONE: 11/05/2014]_
+  - port the stats generation from QuizQuestion::Base#stats into the generic answer analyzer
+  - question statistics support:
+    + Essay _[DONE: 29/04/2014]_
+    + Fill in Multiple Blanks
+    + Fill in The Blank (`short_answer`)
+    + Matching
+    + Multiple Answers
+    + Multiple Choice
+    + Multiple Dropdowns
+    + Numerical
+    + True/False

gems/canvas_quiz_statistics/canvas_quiz_statistics.gemspec

@@ -0,0 +1,23 @@
+# coding: utf-8
+
+require File.join(%W[#{File.dirname(__FILE__)} lib canvas_quiz_statistics version])
+
+Gem::Specification.new do |spec|
+  spec.name          = 'canvas_quiz_statistics'
+  spec.version       = CanvasQuizStatistics::VERSION
+  spec.authors       = ['Ahmad Amireh']
+  spec.email         = ['ahmad@instructure.com']
+  spec.summary       = %q{Bundle of statistics generators for quizzes and quiz questions.}
+  spec.files         = Dir.glob("lib/**/*") + %w[ LICENSE.txt README.md Rakefile ]
+  spec.test_files    = spec.files.grep(%r{spec})
+  spec.require_paths = ['lib']
+
+  spec.add_dependency 'activesupport'
+
+  spec.add_development_dependency 'bundler', '~> 1.5'
+  spec.add_development_dependency 'rake'
+  spec.add_development_dependency 'rspec'
+  spec.add_development_dependency 'guard'
+  spec.add_development_dependency 'guard-rspec'
+  spec.add_development_dependency 'terminal-notifier-guard'
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics.rb

@@ -0,0 +1,5 @@
+module CanvasQuizStatistics
+  require 'canvas_quiz_statistics/version'
+  require 'canvas_quiz_statistics/answer_analyzers'
+  require 'canvas_quiz_statistics/question_analyzer'
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics/answer_analyzers.rb

@@ -0,0 +1,39 @@
+#
+# Copyright (C) 2014 Instructure, Inc.
+#
+# This file is part of Canvas.
+#
+# Canvas is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, version 3 of the License.
+#
+# Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+require 'canvas_quiz_statistics/answer_analyzers/util'
+require 'canvas_quiz_statistics/answer_analyzers/base'
+require 'canvas_quiz_statistics/answer_analyzers/essay'
+
+module CanvasQuizStatistics::AnswerAnalyzers
+  # Convenient access to analyzer for a given question type, e.g:
+  #
+  #   AnswerAnalyzers['multiple_choice_question'].new
+  #
+  # If the question type is not supported, you will be given the Base
+  # analyzer which really does nothing.
+  def self.[](question_type)
+    AVAILABLE_ANALYZERS[question_type] || Base
+  end
+
+  private
+
+  # for fast lookup without having to use #constantize or anything
+  AVAILABLE_ANALYZERS = {
+    'essay_question' => Essay
+  }
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics/answer_analyzers/base.rb

@@ -0,0 +1,31 @@
+#
+# Copyright (C) 2014 Instructure, Inc.
+#
+# This file is part of Canvas.
+#
+# Canvas is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, version 3 of the License.
+#
+# Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+module CanvasQuizStatistics::AnswerAnalyzers
+  class Base
+    def run(question_data, responses)
+      {}
+    end
+
+    # Test whether the response contains an answer to the question.
+    #
+    # Default behavior is to text whether the "text" field is populated.
+    def answer_present?(response)
+      response[:text].present?
+    end
+  end
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics/answer_analyzers/essay.rb

@@ -0,0 +1,70 @@
+#
+# Copyright (C) 2014 Instructure, Inc.
+#
+# This file is part of Canvas.
+#
+# Canvas is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, version 3 of the License.
+#
+# Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+module CanvasQuizStatistics::AnswerAnalyzers
+  # Generates statistics for a set of student responses to an essay question.
+  class Essay < Base
+    # @param [Array<Hash>] responses
+    #   Set of student responses. Entry is expected to look something like this:
+    #   {
+    #     "correct": "undefined",
+    #     "points": 0,
+    #     "question_id": 18,
+    #     "text": "<p>*grunts*</p>",
+    #     "user_id": 4
+    #   }
+    #
+    # @example output
+    # {
+    #   // The number of students whose responses were graded by the teacher so
+    #   // far.
+    #   "graded": 1,
+    #
+    #   // The number of students who got graded with a full score.
+    #   "full_credit": 1,
+    #
+    #   // A set of vectors of scores and the number of students who received
+    #   // each score.
+    #   "point_distribution": [
+    #     { "score": 0, "count": 1 },
+    #     { "score": 3, "count": 1 }
+    #   ]
+    # }
+    def run(question_data, responses)
+      full_credit = question_data[:points_possible]
+
+      stats = {}
+      stats[:graded] = responses.select { |r| r[:correct] == 'defined' }.length
+
+      stats[:full_credit] = responses.select do |response|
+        response[:points] == full_credit
+      end.length
+
+      point_distribution = Hash.new(0).tap do |vector|
+        responses.each { |response| vector[response[:points]] += 1 }
+      end
+
+      stats[:point_distribution] = point_distribution.keys.map do |score|
+        { score: score, count: point_distribution[score] }
+      end
+
+      stats[:point_distribution].sort_by! { |v| v[:score] || -1 }
+
+      stats
+    end
+  end
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics/answer_analyzers/util.rb

@@ -0,0 +1,24 @@
+#
+# Copyright (C) 2014 Instructure, Inc.
+#
+# This file is part of Canvas.
+#
+# Canvas is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, version 3 of the License.
+#
+# Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+module CanvasQuizStatistics::AnswerAnalyzers
+  module Util
+    def self.digest(str)
+      Digest::MD5.hexdigest((str || '').to_s.strip)
+    end
+  end
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics/question_analyzer.rb

@@ -0,0 +1,61 @@
+#
+# Copyright (C) 2014 Instructure, Inc.
+#
+# This file is part of Canvas.
+#
+# Canvas is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, version 3 of the License.
+#
+# Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+require 'active_support/core_ext'
+
+class CanvasQuizStatistics::QuestionAnalyzer
+  attr_reader :question_data
+
+  def initialize(question_data)
+    @question_data = question_data
+    @answer_analyzer = AnswerAnalyzers[question_data[:question_type].to_s].new
+  end
+
+  # Gathers all types of stats from a set of student responses.
+  #
+  # The output will contain the output of an answer analyzer.
+  #
+  # @return [Hash]
+  #   {
+  #     // Number of students who have answered this question.
+  #     "responses": 2,
+  #
+  #     // IDs of those students.
+  #     "user_ids": [ 1, 133 ]
+  #   }
+  def run(responses)
+    {}.tap do |stats|
+      stats.merge! @answer_analyzer.run(question_data, responses)
+      stats.merge! count_filled_responses(responses)
+    end
+  end
+
+  private
+
+  AnswerAnalyzers = CanvasQuizStatistics::AnswerAnalyzers
+
+  # Returns the number of and IDs of students who provided any kind of answer,
+  # regardless of whether it's correct or not.
+  def count_filled_responses(responses)
+    answers = responses.select do |response|
+      @answer_analyzer.answer_present?(response)
+    end
+
+    { responses: answers.size, user_ids: answers.map { |a| a[:user_id] }.uniq }
+  end
+end

gems/canvas_quiz_statistics/lib/canvas_quiz_statistics/version.rb

@@ -0,0 +1,20 @@
+#
+# Copyright (C) 2014 Instructure, Inc.
+#
+# This file is part of Canvas.
+#
+# Canvas is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, version 3 of the License.
+#
+# Canvas is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+module CanvasQuizStatistics
+  VERSION = '0.1.0'
+end

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/answer_analyzers/essay_spec.rb

@@ -0,0 +1,68 @@
+require 'spec_helper'
+
+describe CanvasQuizStatistics::AnswerAnalyzers::Essay do
+  let(:question_data) { QuestionHelpers.fixture('essay_question') }
+
+  it 'should not blow up when no responses are provided' do
+    expect {
+      subject.run(question_data, []).should be_present
+    }.to_not raise_error
+  end
+
+  it "should not consider an answer to be present if it's empty" do
+    subject.answer_present?({ text: nil }).should be_false
+    subject.answer_present?({ text: '' }).should be_false
+  end
+
+  describe 'output [#run]' do
+    describe ':full_credit' do
+      it 'should count all students who received full credit' do
+        output = subject.run(question_data, [
+          { points: 3 }, { points: 2 }, { points: 3 }
+        ])
+
+        output[:full_credit].should == 2
+      end
+
+      it 'should be 0 otherwise' do
+        output = subject.run(question_data, [
+          { points: 1 }
+        ])
+
+        output[:full_credit].should == 0
+      end
+    end
+
+    it ':graded - should reflect the number of graded answers' do
+      output = subject.run(question_data, [
+        { correct: 'defined' }, { correct: 'undefined' }
+      ])
+
+      output[:graded].should == 1
+    end
+
+    describe ':point_distribution' do
+      it 'should map each score to the number of receivers' do
+        output = subject.run(question_data, [
+          { points: 1, user_id: 1 },
+          { points: 3, user_id: 2 }, { points: 3, user_id: 3 },
+          { points: nil, user_id: 5 }
+        ])
+
+        output[:point_distribution].should include({ score: nil, count: 1 })
+        output[:point_distribution].should include({ score: 1, count: 1 })
+        output[:point_distribution].should include({ score: 3, count: 2 })
+      end
+
+      it 'should sort them in score ascending mode' do
+        output = subject.run(question_data, [
+          { points: 3, user_id: 2 }, { points: 3, user_id: 3 },
+          { points: 1, user_id: 1 },
+          { points: nil, user_id: 5 }
+        ])
+
+        output[:point_distribution].map { |v| v[:score] }.should == [ nil, 1, 3 ]
+      end
+    end
+  end
+end

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/answer_analyzers/generic_spec.rb

@@ -0,0 +1,17 @@
+require 'spec_helper'
+
+describe CanvasQuizStatistics::AnswerAnalyzers::Base do
+  describe '#run' do
+    it 'returns an empty set' do
+      subject.run({}, []).should == {}
+    end
+  end
+
+  describe '#answer_present?' do
+    it 'defaults to testing whether :text is present' do
+      subject.answer_present?({ text: 'foo' }).should be_true
+      subject.answer_present?({}).should be_false
+      subject.answer_present?({ text: nil }).should be_false
+    end
+  end
+end

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/answer_analyzers_spec.rb

@@ -0,0 +1,15 @@
+require 'spec_helper'
+
+describe CanvasQuizStatistics::AnswerAnalyzers do
+  Analyzers = CanvasQuizStatistics::AnswerAnalyzers
+
+  describe '[]' do
+    it 'should locate an analyzer' do
+      subject['essay_question'].should == Analyzers::Essay
+    end
+
+    it 'should return the generic analyzer for questions of unsupported types' do
+      subject['text_only_question'].should == Analyzers::Base
+    end
+  end
+end

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/question_analyzer_spec.rb

@@ -0,0 +1,43 @@
+require 'spec_helper'
+
+describe CanvasQuizStatistics::QuestionAnalyzer do
+  subject do
+    described_class.new(@question_data || { })
+  end
+
+  it 'should not break with no responses' do
+    expect { subject.run([]) }.to_not raise_error
+  end
+
+  it 'should embed the output of the answer analyzer' do
+    @question_data = {
+      question_type: 'stubbed_question'
+    }
+    responses = [{ user_id: 1, text: 'foobar' }]
+    answer_analysis = { some_metric: 12 }
+
+    analyzer = double
+    analyzer.should_receive(:run).with(@question_data, responses).and_return(answer_analysis)
+    analyzer.should_receive(:answer_present?).and_return(true)
+
+    analyzer_generator = double
+    analyzer_generator.should_receive(:new).and_return(analyzer)
+    CanvasQuizStatistics::AnswerAnalyzers.stub :[] => analyzer_generator
+
+    output = subject.run(responses)
+    output[:some_metric].should == 12
+  end
+
+  describe ':answered' do
+    it ':count - the number of students who provided an answer' do
+      output = subject.run([
+        { text: 'hi', user_id: 1 },
+        { text: 'hello', user_id: 3 }
+      ])
+
+      # answer_analyzer.stub(:answer_present?).and_return(true)
+      output[:responses].should == 2
+      output[:user_ids].should == [1,3]
+    end
+  end
+end

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/support/fixtures/essay_question_data.json

@@ -0,0 +1,18 @@
+{
+  "id": 48,
+  "regrade_option": false,
+  "points_possible": 3,
+  "correct_comments": "",
+  "incorrect_comments": "",
+  "neutral_comments": "",
+  "question_type": "essay_question",
+  "question_name": "[Essay] Question",
+  "name": "[Essay] Question",
+  "question_text": "<p>[Essay] Summarize your feelings towards life, the universe, and everything in decimal numbers.</p>",
+  "answers": [],
+  "text_after_answers": "",
+  "comments": "",
+  "assessment_question_id": 45,
+  "position": 6,
+  "published_at": "2014-04-15T04:37:02Z"
+}

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/support/fixtures/multiple_choice_question_data.json

@@ -0,0 +1,44 @@
+{
+  "id": 1,
+  "regrade_option": false,
+  "points_possible": 1,
+  "correct_comments": "",
+  "incorrect_comments": "",
+  "neutral_comments": "",
+  "question_type": "multiple_choice_question",
+  "question_name": "MC Question",
+  "name": "MC Question",
+  "question_text": "<p>[MC] A, B, C, or D?</p>",
+  "answers": [
+    {
+      "id": 3023,
+      "text": "A",
+      "html": "",
+      "comments": "",
+      "weight": 100
+    },
+    {
+      "id": 8899,
+      "text": "B",
+      "html": "",
+      "comments": "",
+      "weight": 0
+    },
+    {
+      "id": 7907,
+      "text": "C",
+      "html": "",
+      "comments": "",
+      "weight": 0
+    },
+    {
+      "id": 5646,
+      "text": "D",
+      "html": "",
+      "comments": "",
+      "weight": 0
+    }
+  ],
+  "text_after_answers": "",
+  "assessment_question_id": null
+}

gems/canvas_quiz_statistics/spec/canvas_quiz_statistics/support/question_helpers.rb

@@ -0,0 +1,27 @@
+require 'json'
+
+module QuestionHelpers
+  FixturePath = File.join(File.dirname(__FILE__), 'fixtures')
+
+  # Loads a question data fixture from support/fixtures/*_data.json, just pass
+  # it the type of the question, e.g:
+  #
+  #   @question_data = question_data_fixture('multiple_choice_question')
+  #   # now you have a valid question data to analyze.
+  #
+  #   # and you can munge and customize it:
+  #   @question_data[:answers].each { |a| ... }
+  #
+  # @return [Hash]
+  def self.fixture(question_type)
+    path = File.join(FixturePath, "#{question_type}_data.json")
+
+    unless File.exists?(path)
+      raise '' <<
+        "Missing question data fixture for question of type #{question_type}" <<
+        ", expected file to be located at #{path}"
+    end
+
+    JSON.parse(File.read(path)).with_indifferent_access
+  end
+end

gems/canvas_quiz_statistics/spec/spec_helper.rb

@@ -0,0 +1,29 @@
+require 'simplecov'
+require 'simplecov-rcov'
+
+SimpleCov.use_merging
+SimpleCov.merge_timeout(10000)
+SimpleCov.command_name('canvas-quiz-statistics-gem')
+SimpleCov.start('test_frameworks') do
+  SimpleCov.coverage_dir(File.join(File.dirname(__FILE__), '..', 'coverage'))
+end
+
+require 'canvas_quiz_statistics'
+
+RSpec.configure do |config|
+  config.treat_symbols_as_metadata_keys_with_true_values = true
+  config.run_all_when_everything_filtered = true
+  config.filter_run :focus
+  config.color = true
+  config.order = 'random'
+
+  support_files = File.join(
+    File.dirname(__FILE__),
+    'canvas_quiz_statistics',
+    'support',
+    '**',
+    '*.rb'
+  )
+
+  Dir.glob(support_files).each { |file| require file }
+end

spec/models/quizzes/quiz_statistics/student_analysis_spec.rb

@@ -316,4 +316,25 @@ describe Quizzes::QuizStatistics::StudentAnalysis do
     stats.last[9].should == "lolcats,lolrus"
   end
 
+  describe 'question statistics' do
+    subject { Quizzes::QuizStatistics::StudentAnalysis.new({}) }
+
+    it 'should proxy to the gem for Essay question stats' do
+      question_data = { question_type: 'essay_question' }
+      responses = []
+      stats = { foo: 'bar' }
+      analyzer = stub
+
+      CanvasQuizStatistics::QuestionAnalyzer.
+        expects(:new).with(question_data).
+        returns(analyzer)
+
+      analyzer.expects(:run).with(responses).returns(stats)
+      output = subject.send(:stats_for_question, question_data, responses)
+      output.should == {
+        question_type: 'essay_question',
+        foo: 'bar'
+      }.with_indifferent_access
+    end
+  end
 end