fixes CNVS-15153
* don't raise if the configured cassandra database can't be connected
to; instead, return nil (uncached, so it can be retried next time)
and treat it as connected but unavailable
* allow configuring whether an error is re-raised (configured true for
test only in canvas), rather than peeking into rails directly from
event stream.
* allow registering callbacks to run on insert/update error via
on_error
* on insert/update error, before (maybe) reraising the error, call any
registered callbacks
* configure canvas to log to Rails.logger and statsd (removed from
EventStream::Failure) on insert/update error
EventStream::Failure is now unused, but not yet removed from the code.
it should be considered deprecated and remain unused; it will be removed
in the near future.
test-plan:
- start canvas in non-test mode (e.g. development mode) with cassandra
and statsd configured and running
- turn off cassandra, so even stream inserts should fail
- change a grade (to trigger a GradeChange event)
- with cassandra off, event insert should fail, but should *not*
impede the remainder of the grade change action
- EventStream::Failure record should *not* be created in the database
- an error line should appear in the Rails log for the failed insert
- the appropriate 'event_stream_failure.*' statsd counters should be
incremented
Change-Id: I5c6d29c2a08276ccc06ebc8c1a59e0d33ce2cc4e
Reviewed-on: https://gerrit.instructure.com/40088
QA-Review: August Thornton <august@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
954e77d645