and rename that suite of methods to make more sense
test-plan: N/A
Change-Id: Iffc520ea55141ac47da669663838a4d3c3d8712c
Reviewed-on: https://gerrit.instructure.com/142486
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
Closes: CNVS-39627, AMS-701
The Delayed::Worker around callbacks are chained,
so all of the required parameters must be passed
through to the block in case it is another callback
that relies on them.
Change-Id: I740d42bfc36e3dda52c85802ddca673f2b2f6bb8
Reviewed-on: https://gerrit.instructure.com/128232
Tested-by: Jenkins
Reviewed-by: Dave Donahue <ddonahue@instructure.com>
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Product-Review: Jeremy Slade <jslade@instructure.com>
QA-Review: Jeremy Slade <jslade@instructure.com>
fixes CNVS-39296
test plan:
* run jobs/web (must be using a web server that doesn't capture HUP
itself)
* HUP your processes
* notice that they don't crash, and the logs will say that the reloader
ran
Change-Id: I150729fdb68c0fd0ff462d794eecfc8da7e17e5e
Reviewed-on: https://gerrit.instructure.com/125968
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
goal is to make all statsd keys for jobs "*.class.method.*" so we can
easily parameterize queries across the keyspace.
ruby-statsd by default creates namespaces for ruby modules, (see
https://github.com/reinh/statsd/blob/v1.0.0/lib/statsd.rb#L138), so we
want to prevent that behavior.
closes CNVS-34003
test plan:
- connect canvas to statsd
- run some jobs
- they should all show up with the namespaces as mentioned above
Change-Id: Ica7c2c11c50537539ed5a3169f38b9f97d587919
Reviewed-on: https://gerrit.instructure.com/98223
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
Refs: PLAT-2369
Test-plan:
- setup multiple root accounts with /c/106174/
- spinup local live-event testing services
- test out live-event spawning processes as a result of a delayed job
- assert that the kinesis stream contains events with attributes
containing `root_account_uuid`
Change-Id: Ia00dd8d25497529dcf59b9d7b3d0b438d334a61e
Reviewed-on: https://gerrit.instructure.com/106175
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jayce Higgins <jhiggins@instructure.com>
fixes CNVS-34638
test plan:
* add `work_queue: parent_process` to your delayed_jobs.yml
* run (multiple) jobs in the background
* jobs should process
Change-Id: I087540c8319120092384bd0533af7e770f1dc1af
Reviewed-on: https://gerrit.instructure.com/102024
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
fixes CNVS-29727
test plan:
- enable live events
- ensure that all global ids come through as strings
Change-Id: I54d1105d442dc92b0d973777fd745864bea1f232
Reviewed-on: https://gerrit.instructure.com/81428
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Benjamin Christian Nelson <bcnelson@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
fixes CNVS-274522
includes a migration to detect if you were using it, and fold it into
the main database
Change-Id: I1afbbf5f91c4fb4165a3e6dc484d318edc390255
Reviewed-on: https://gerrit.instructure.com/72811
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
test plan:
- perform an action that will generate a delayed job
- the last line of the delayed job log, which starts with STAT, should
include the job_id
Change-Id: I3c02f01494ffa138fa4f5dc022522d8c7f3ab2ef
Reviewed-on: https://gerrit.instructure.com/64540
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
closes CNVS-6016
No more error reports! (soon)
this commit builds up sentry integration through the new
Canvas::Errors module, along with other things that need
to happen on every exception. ErrorReports
should now get pushed towards just being used for representing
a complaint a user filed via the get help form.
I fixed about half the things that got linted as well
while I was in here, but because this touches to much
I fear divergence from tackling too many (I think we
can safely say it's "better than we found it")
I left a lot of the infrastructure for error reports in place
until other commits for plugins can be merged
TEST PLAN:
1) setup your raven.yml config file with the dsn for our
sentry install
2) force an error to happen in a request response cycle.
3) see the error in sentry
4) force an error to happen in a job
5) see the error in sentry
6) statsd increments shoudl still fire
7) for the moment, an error report should still get created.
Change-Id: I5a9dc7214598f8d5083451fd15f0423f8f939034
Reviewed-on: https://gerrit.instructure.com/51621
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins
QA-Review: August Thornton <august@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
This commit adds a new module called LiveEvents that knows how to send a
certain set of events to Kinesis. The module is configured via
normal plugin settings per account. Once the plugin is configured with
a Kinesis stream, events will start getting sent to that stream.
Events are sent asynchronously, in a background thread.
test plan:
* See `doc/live_events.md` for instructions on how to setup a local
kinesis stream and configure the LiveEvents plugin.
* Start tailing the stream with the command specified in
`doc/live_events.md` in a terminal.
* Perform the actions described in `doc/api/live_events.md` and verify
that events show up in your Kinesis terminal with the correct data.
Change-Id: Id799688c972205a1eee84a673912f84b0c7abb57
Reviewed-on: https://gerrit.instructure.com/50324
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
Product-Review: Zach Wily <zach@instructure.com>
closes CNVS-15881
This mostly involved adding hooks to the gem for things that we were
directly modifying in Canvas, then hooking into those points in the
canvas config/initializers/delayed_job.rb file.
Periodic jobs scheduling changed a bit as well -- there's no longer
support in the extracted gem for reading a special file under config/,
so I moved our periodic_jobs.rb file to a normal initializer.
test plan: delayed jobs should still work as before, including queuing
jobs, running the worker pool, and in tests.
Change-Id: I9ce57091d18f21f4355011fcb75230193d53facb
Reviewed-on: https://gerrit.instructure.com/42027
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
test plan:
- regression test to make sure that you can navigate to different
domains and shards, create + process delayed jobs, and navigate to the
default and site admin accounts
Change-Id: I0ed4bf3309e3950a046ed1cd7bc7b1fe0a806a7a
Reviewed-on: https://gerrit.instructure.com/39972
Tested-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
fixes: CNVS-11425
This is a performance refactor of the permissions. The
biggest change is the caching. The cache key is now based
on the right so each right will be cached by itself. The
goal is to reduce places where we implement caching for
permissions and let adheres_to_policy handle it.
Another commit is coming to clean up calls to the new
methods created here. g/34280
Test Plan:
- Make sure permissions all work still.
- Make masquerading still works with permissions.
- Make sure switing views such as "student view" for a
course.
Change-Id: I4a30b0aba394cea24c3b60167fc1369a2584f5a4
Reviewed-on: https://gerrit.instructure.com/34278
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
refs CNVS-13024
Setting wasn't properly being initialized as unsharded because
it was loading before Switchman. The reason we need Setting before
switchman is just for yaml loading, so split that into its own
class.
Change-Id: I5456e103cb216dba2d5af4e9c20a697b468c923b
Reviewed-on: https://gerrit.instructure.com/35043
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
closes CNVS-12739
log timing for pops, time spent in queue, time to perform job,
sliced by tag (where applicable), shard, and job shard.
test plan:
* configure statsd.yml to go to 127.0.0.1 port 7856
* nc -lu 127.0.0.1 7856
* run jobs
* you should see lots of job timings sent to netcat
Change-Id: I60a4b6b10d30edd96011e8e8bc8aa6104dfc6daa
Reviewed-on: https://gerrit.instructure.com/34724
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
This logs the STAT line in the same format as for web requests, though
omitting the @@last_sample at the end.
Rather than implementing it directly in worker.rb, I pulled in the new
Lifecycle callback code that delayed_job added after we forked, and
added the logging as an around filter in our canvas initializer.
Eventually I'd like to pull the other canvas-specific code into
lifecycle callbacks as well, making delayed_job more generic and
extractable.
test plan: run jobs, and verify that they still complete successfully,
and log a STAT line at the end with information on cpu and memory usage.
Change-Id: I18e17b286f5df8aa3c9774f0ac92841f4b92e193
Reviewed-on: https://gerrit.instructure.com/30037
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
This worked in 1.8 but breaks in 1.9
test plan: create a delayed job that'll fail -- for instance
> User.find(X).send_later(:name)
> User.find(X).destroy!
Then run jobs, and the job should get inserted into failed_jobs
correctly.
Change-Id: Ie3a1f28cfaa7f1b1032b1c26dcee3330c30c0a5e
Reviewed-on: https://gerrit.instructure.com/17906
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
This supports all the features of our current delayed jobs backend,
including priorities, queues, jobs scheduled for the future, strands,
periodic jobs, etc.
Atomicity is guaranteed by using the new Lua scripting features of Redis
2.6, so this requires 2.6 (which is still only a release candidate as of this
time).
It's not yet production ready (see the TODO at the top of the
redis/job.rb file), so trying to start canvas in prod with redis jobs
configured will raise an error as a safeguard.
The tests have been modified to all pass with redis configured as the
jobs backend. Some selenium specs were removed because the jobs UI lost
some functionality that I didn't think was important enough to port to
redis (listing the most popular tags for future and failed jobs, for
example).
Change-Id: Ie57b15bae1d4ba7b2b2344c872411165551d1ac8
Reviewed-on: https://gerrit.instructure.com/12120
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
test plan:
* don't start delayed jobs
* go to your local jobs page: http://localhost:3000/jobs
* open script/console and type: "now or later".send_now_or_later :later, :to_s
* refresh the "Jobs List" and see if a String#to_s job appeared
* start DJ and refresh to make sure the job was run
Change-Id: Iab997a4fcb252b481774309194dc6a0e937ecf4a
Reviewed-on: https://gerrit.instructure.com/7194
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
Assessment questions can be shared among multiple contexts, so we need to make
the Canvas links used inside authorized to anyone with the link.
Change-Id: I0df4907405fd518ee0efebccf1b9fb8717dca141
Reviewed-on: https://gerrit.instructure.com/6341
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
optimize crosslisting itself:
* use ids where possible to avoid unnecessarily loading up objects
* Don't worry about keeping track of if we need to save; we're gonna
save anyway
* update account associations on any account change, not just root
account change (if you re-crosslist a section from one sub-account
to another sub-account, the users may no longer be associated with
the first sub-account).
optimize sis imports:
* really batch up update_account_associations
Change-Id: Ic0fbe1601afcbcd3e6540e69febc2e6a1a94157f
Reviewed-on: https://gerrit.instructure.com/5137
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
and some work on delayed_jobs refactoring
refs #4226
Change-Id: I21a91a44368e77aef4a75e0d30cefe252a901691
Reviewed-on: https://gerrit.instructure.com/3640
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Jacob Fugal