fixes CNVS-34832
the arguments to these methods have not changed, just their names. and the
new methods became available in Rails 4, so we can start using them now
and prevent someone from doing old-style in the future
Change-Id: I61aa5512995dc8f25f3f7bd009a6cfa0a030e274
Reviewed-on: https://gerrit.instructure.com/101401
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
This patch exposes delayed jobs data as an API route, which makes it accessible using a bearer token. Currently, a token is not accepted as valid authentication, and an Unauthorized error is returned to the client.
Test plan:
- create a user in the Site Admin account and grant it admin rights to Site Admin
- create a user in the Site Admin account (or any other account) but do not grant it admin rights to Site Admin
- create some delayed jobs, failed and otherwise
- test the following routes:
-`GET /api/v1/jobs?only=jobs&flavor=[failed|future|current|waiting]`: should return list of jobs for specified flavor in JSON
-`GET /api/v1/jobs/:id`: should return a specific non-failed job
-`GET /api/v1/jobs/:id`: should return a specific failed job
- `POST /api/v1/jobs/batch_update` with body
`update_action=destroy&job_ids[]=$id1&job_ids[]=$id2`: (replacing
$id1 and $id2 with the IDs of real non-failed jobs) should return a success message with count of destroyed jobs.
- `POST /api/v1/jobs/batch_update` with body
`update_action=destroy&job_ids[]=$id1&job_ids[]=$id2&flavor=failed`:
(replacing $id1 and $id2 with the IDs of real failed jobs) will fail
pending instructure/canvas-jobs#2 (with that pull, it will return a
success message with count of destroyed jobs).
- `POST /api/v1/jobs/batch_update` with body `update_action=destroy&flavor=future`: should return a success message with count of destroyed jobs
- `POST /api/v1/jobs/batch_update` with body
`update_action=hold&flavor=future`: should return an error report
(can't hold failed jobs).
When testing the routes, using the second user created (without Site Admin rights) should throw an Unauthorized error. Using a developer key should throw an Invalid Token error.
refs CNVS-7597
render :json => thing will call ActiveSupport::JSON.encode(thing) unless
thing is a String. ActiveSupport::JSON.encode(thing) just calls
thing.to_json but with some circular reference checking that we want. we
may also want enhance ActiveSupport::JSON.encode to do additional
processing, and calling to_json straight up would bypass that.
in the cases where we do need to do the structural transformation before
passing to render :json (e.g. because of need to pass arguments), use
as_json to do structural transformation only, vs. to_json that does
serialization of the as_json result.
adds a rake task to lint the controllers to enforce as_json over to_json
in render json calls.
test-plan: heavy regression testing; no end behavior should change
(except a pair of serialization bugs that got fixed)
Change-Id: I7a91a9fe0eca70456bc5bca233f0ed6b27a54aaf
Reviewed-on: https://gerrit.instructure.com/23650
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
extracted out of canvas-lms
refs CNVS-4713
test plan:
* actions that use a slave should still work (dashboard render)
* you should be able to switch envs and users in console
Change-Id: I07dda8057cf94383bc4579f1ef6b5a4b3ffc20b5
Reviewed-on: https://gerrit.instructure.com/19287
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
test plan:
* create a site admin role with only view jobs permission
* as a user in that role, you should be able to see the jobs page,
but not hold, un-hold, or delete jobs
Change-Id: I5438ef55d13f87773b9e664f87e5b0d167ad4767
Reviewed-on: https://gerrit.instructure.com/16463
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
The change to load these details on demand didn't know how to properly
handle failed jobs.
test plan: go to /jobs as a site admin, filter by failed jobs, select
one, you should be able to view the handler and last error details.
Change-Id: Ibb7cdd768726a4952d29d3185f8e4524eae3f064
Reviewed-on: https://gerrit.instructure.com/15595
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
rather than loading these fields for every job (they can be huge,
especially handler)
test plan:
* visit the jobs page
* ensure running jobs list and jobs list refresh automatically
* select a job, then click to view the handler or last error, it should
load and display
Change-Id: Id371670ad39d16536903eac882ce9f4087eeaffb
Reviewed-on: https://gerrit.instructure.com/14416
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
A set of class functions were added to Delayed::Backend::ActiveRecord
for all the querying a updating functionality that the jobs admin needs,
so that no direct ActiveRecord queries are needed. The /jobs UI is
refactored to use these new functions.
There are a few differences in behavior: The search isn't a combined
wildcard search anymore. Instead, new "flavors" were added to the
drop-down for strand, tag, and ID. The search box searches only the
selected attribute, and it's exact match now.
Specs are being updated to use these new functions as well. Eventually,
no direct AR queries will be done against Jobs anywhere, so that non-AR
jobs backends are possible.
Also as part of this, all jobs require a queue now. Passing nil for the
queue will use the default of Delayed::Worker.queue.
test plan: Load /jobs, and verify that it works as before except where
there are differences as described above.
* Selecting flavors of jobs lists only those jobs.
* Searching by ID, strand or tag works.
* The hold/unhold/delete actions work in the various combinations of
filtering/searching.
* Linking to an individual job still works (though the query string
has changed so old links don't work)
* Running jobs and list of popular tags still works as expected.
Change-Id: Iffd5b8c7b3d6e4b128792a9dee7b97c6dfb251dc
Reviewed-on: https://gerrit.instructure.com/12632
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
test plan: go to /accounts/X (where X is Account.site_admin), and verify
the Plugins and Jobs tabs show on the left if you have the correct
permissions. Go to both those pages and verify the breadcrumbs.
Change-Id: Id63492cf0cc753151750570106cc87b5af2d73f7
Reviewed-on: https://gerrit.instructure.com/8204
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
* hold/unhold/delete in one query, instead of loading each job and then saving it
* fix a long-standing bug where searching for a job by id and then
acting on it, would actually act on all jobs (holding all jobs, etc)
* tweak the timeouts/sleeps on jobs specs to remove about 10s from their runtime
refs #6875
Change-Id: I4db5df1a938bf60b95b9972ce8891adcdc16f4f5
Reviewed-on: https://gerrit.instructure.com/8210
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
This was already a small issue if the job queue was on a different
database driver than the main database, and it'll become more important
as more AR connections are introduced.
Change-Id: I204becadd32bb935df096e8c937a04bb6962f0b2
Reviewed-on: https://gerrit.instructure.com/4601
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
To simplify the queries done on jobs:
* failed jobs now go in a separate table
* jobs that go past their max_run_time don't immediately get re-run.
instead a periodic job goes through every 5 minutes and unlocks
expired jobs
* strand performance is much improved
* we filter out jobs that can't be run because of strand
restrictions completely in the database query now
* better indexes help with these queries and the sub-select
Change-Id: Ida494da2fcec409bff330b7676c9a6b9413b3a06
Reviewed-on: https://gerrit.instructure.com/4163
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Though if the job completes, it'll get destroyed, so "perma" for some
value of permanent.
Change-Id: I1c158083920789040971470dad4a7fdb81997922
Reviewed-on: https://gerrit.instructure.com/3978
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
and some work on delayed_jobs refactoring
refs #4226
Change-Id: I21a91a44368e77aef4a75e0d30cefe252a901691
Reviewed-on: https://gerrit.instructure.com/3640
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Landon Wilkins