closes CNVS-12739
log timing for pops, time spent in queue, time to perform job,
sliced by tag (where applicable), shard, and job shard.
test plan:
* configure statsd.yml to go to 127.0.0.1 port 7856
* nc -lu 127.0.0.1 7856
* run jobs
* you should see lots of job timings sent to netcat
Change-Id: I60a4b6b10d30edd96011e8e8bc8aa6104dfc6daa
Reviewed-on: https://gerrit.instructure.com/34724
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
* cache entries are wrapped in rails 3
* serialized attributes are handled differently
* stub out a bunch of classes that will just be ignored
on the returned object
Change-Id: Iff15a0f5c39cd7ceae25144585539acbfc7a62ad
Reviewed-on: https://gerrit.instructure.com/31717
Reviewed-by: James Williams <jamesw@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
test plan
- run migrations with just canvas-lms
- migrations should not fail
Change-Id: I489fd5c89457771ae2d21a7d110274d3843b3062
Reviewed-on: https://gerrit.instructure.com/31264
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Change-Id: I0bc5b04d6210c90aa607e23714e7e78a2964e8ff
Reviewed-on: https://gerrit.instructure.com/29517
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Change-Id: Ib2147e2a5c388ba56c1e558b414da9b7ec4c9cd2
Reviewed-on: https://gerrit.instructure.com/27614
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
fixes CNVS-9085
test plan:
* login using site admin credentials at the default account
* via the UI, enroll a site admin via login, sis id, and e-mail
address in a course
Change-Id: If525949d4da780e4750ecf53f7acd8aa5440387e
Reviewed-on: https://gerrit.instructure.com/26729
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
rather than CANVAS_RAILS3 or Rails.version
this is to be consistent, and to reinforce that any "special" branches
are for rails 2.3 backwards compatibility while trying to target rails
3, rather than rails 3 "forwards compatibility".
Change-Id: I4494b65e3f71108a43d09032c1569c478646a828
Reviewed-on: https://gerrit.instructure.com/24998
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
This can successfully load rails console and rails server. There are
many, many problems still. The idea is this won't change anything under
rails 2.3, it's all backwards compatible.
closes CNVS-4711
test plan: `touch RAILS3` in your Canvas Rails.root directory. The run
`bundle update` and verify that you get rails 3 installed. Run `bundle
exec rails c` to load console or `bundle exec rails s` to start a
webrick server. You can login, though the dashboard currently breaks.
Also jammit isn't working yet.
But more importantly, Rails 2.3 should still work same as ever. All
tests should pass, and a basic regression sanity check would be good too.
Change-Id: Idd6f35de88adde84cd2db3a650f44b71bd6e9684
Reviewed-on: https://gerrit.instructure.com/18453
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
fixes CNVS-4625
test-plan:
- create four shards A, B, C, and D.
- create three users A, B, and C on shards A, B, and C, respectively.
- create conversation on shard A between users A and B:
- on shard A as user A, filter by user B
- on shard B as user A, filter by user B
- on shard C as user A, filter by user B
- on shard B as user B, filter by user A
- on shard C as user B, filter by user A
- create conversation on shard A between users B and C:
- on shard A as user B, filter by user C
- on shard B as user B, filter by user C
- on shard C as user B, filter by user C
- on shard D as user B, filter by user C
- create users D and E on shards A and B, respectively.
- create conversation on shard A between users A and D:
- on shard A as user A, filter by user D
- on shard B as user A, filter by user D
- create conversation on shard A between users B and E:
- on shard B as user B, filter by user E
- on shard C as user B, filter by user E
- create conversation on shard B between users D and E:
- on shard B as user E, filter by user D
- create user F on shard A.
- create conversation on shard B between users A and F:
- on shard B as user A, filter by user F
Change-Id: If41a801f5b6b8a79c8fc0717b10293467ba92c19
Reviewed-on: https://gerrit.instructure.com/18522
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
Separates, streamlines, and makes shard-aware all use cases of
User#messageable_users *other* than searching (call site in
SearchController#matching_participants).
Produces three new methods that take the bulk of that responsibility:
* User#load_messageable_users -- given a set of users, filter out the
ones that aren't messageable, and load any common contexts for those
that are.
* User#load_messageable_user -- as User#load_messageable_users, but for
just one user.
* User#messageable_users_in_context -- given a context (a course,
section, or group), return the list of messageable users in that
context.
refs CNVS-2519
remaining on CNVS-2519 is to tackle the search application of
User#messageable_user. mostly there, but reconciling pagination
with ranking by number of shared contexts is proving problematic, so I'm
going to separate that into another commit.
meanwhile, I've renamed User#messageable_users to
User#deprecated_search_messageable_users to discourage any accidental
new uses and left it otherwise untouched. searching for users on the
same shard should be unaffected. You can still locate messageable users
on other shards to insert into conversations by browsing the shared
contexts.
test-plan:
* create user A in shard X
* create user B in shard Y
* for situations where A could message B if on the same shard:
- setup the situation where the common tie is on shard X (e.g. course
on shard X and both A and B in it). run exercises below
- setup the situation where the common tie is on shard Y. run
exercises.
- if appropriate, setup the situation where the common tie is on
shard Z. run exercises.
* for each setup described above, login as A:
- A should see the "message this user" button on B's profile
- if the common tie is a course, section, or group, A should see B
under that context when the context is selected in the recipient
browser
- if a conversation exists involving both A and B, when A loads the
conversation they should see B tagged with the common contexts
* regression test searching for messageable users from the same shard
Change-Id: Ibba5551f8afc2435fd14a2e827a400bf95eae76a
Reviewed-on: https://gerrit.instructure.com/17569
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
this stub doesn't do anything with them, but specific sharding
implementations can use them, and the signature should allow that.
Change-Id: I11a27e52e1e98e7aa77818dcf5a00d829bdaef41
Reviewed-on: https://gerrit.instructure.com/17578
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
fixes CNVS-1338
basically, just process associations for the current shard, unless
we're explicitly calling update_account_associations on just the user
(since all the other triggers will only affect associations within
the same shard)
test plan:
* enroll an existing site admin user from the default shard in to a
course on another shard
* the user should immediately show up in the course's account's user
list
* remove the user from the course
* the user should immediately disappear from the course's account's
user list
* note the associations a user has
(u.user_account_associations.with_each_shard)
* delete them all (u.user_account_associations.with_each_shard { |s|
s.delete_all })
* call u.update_account_associations
* make sure they all get recreated
Change-Id: I794363d1703fb2d46cd27b67b1216fedef4deaae
Reviewed-on: https://gerrit.instructure.com/17155
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
fixes CNVS-1171
test plan:
* full conversations regression test
* initiate a conversation with a user from another shard
* reply to that conversation from both the sender and the
receiver
* repeat for a group conversation involving two or more
shards
* repeat for huge batch conversations with hundreds of
users and two or more shards
* known NOT working yet:
* re-using the correct cross-shard private conversation
* probably the tagging of messages with Course x,
Group y, etc.
Change-Id: I52549039875941cd518077cea4e28bfd2bc10dbf
Reviewed-on: https://gerrit.instructure.com/16523
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
also change all_pseudonyms and accounts to use the new
HasManyAssociation#with_each_shard hawtness
test plan:
* enroll the same user in courses in multiple shards
* favorites don't work yet; be sure to reset them before
continuing
* courses from all shards should show up in your menu
* courses from all shards should show up in your all courses
page
* when enrolled as a student in a course not on the user's shard,
permissions should be correct (particularly, you should be able
to participate as a student - turn in assignments)
* enroll the user in groups from multiple shards
* all groups should show up in home menu
Change-Id: I84fffa39ebf3dee093ecff145670a29296da29a1
Reviewed-on: https://gerrit.instructure.com/14722
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
* don't use set_shard_override, just set shard= directly
* when creating follows, run the transaction, the search, and
the create all on the same shard
* when searching for upvotes, root on the user, not the
collection item
Change-Id: I5ae3f29e5f48f92c02d3a0c0e04765956d8eb892
Reviewed-on: https://gerrit.instructure.com/14648
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Adds a new back-end store for page_views, using a Cassandra cluster. All
the current page view queries are supported, many using denormalized
views on the data.
test plan:
first, canvas instances that are currently using AR page views
should function as before.
by Setting.set('enable_page_views', 'cassandra') and restarting, you will
switch to cassandra page views. a script to migrate the AR page views to
Cassandra is coming. all page view functionality should work as before.
note that the format of the pagination headers in the
/api/v1/users/X/page_views endpoint has changed.
Change-Id: I2d1feb4d83b06a0c852e49508e85e8dce87507b4
Reviewed-on: https://gerrit.instructure.com/14258
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
test plan:
* add a user from one shard as an admin in an account in another
shard
* that account should show up in the user's home menu, and their
all accounts page
Change-Id: Ib92b1a7f9283f6444d4a59108dc783f583b245bc
Reviewed-on: https://gerrit.instructure.com/14077
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
test plan:
* have a site admin user in shard 1
* add that user by their login id as an admin to an account in
shard 2 (in the UI)
* it should work
* repeat for enrolling in a course
Change-Id: I403f0f853056d4ea1dd9628c70e882fdc3cfd8bf
Reviewed-on: https://gerrit.instructure.com/14090
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
test plan:
* make sure page views are still processed
Change-Id: I07ead6c075823c2021fdcc0cc1f1fddf70f695d0
Reviewed-on: https://gerrit.instructure.com/13810
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
A user can currently follow a Collection, and another User.
Right now following doesn't do anything -- a subsequent commit will add
stream item support for followed Collections.
Also don't scope collection endpoints with /users/X or /groups/Y etc,
except for the index/create actions
test plan: use the api to follow and unfollow users and collections.
verify that you can follow something on a different shard, and it'll
still get returned correctly.
Change-Id: I9fd3f179885327e49f4d220784509de0ea23c0a7
Reviewed-on: https://gerrit.instructure.com/11057
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
including batch jobs
test plan:
* n/a without a sharding implementation
Change-Id: Id4e8b8a72e71a21e0862fda27059f57412a80a53
Reviewed-on: https://gerrit.instructure.com/10165
Reviewed-by: Jon Jensen <jon@instructure.com>
Tested-by: Cody Cutrer <cody@instructure.com>
basically for detailed reports, have a row per account, instead of
stuffing everything into one massive hash with every account
test plan:
* ensure that the account statistics page still works
* ensure that the statistics update when the delayed job runs
Change-Id: I2d4c9ff5a86c8df6d82097aa4f0203a0f55ddb2d
Reviewed-on: https://gerrit.instructure.com/9565
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Cody Cutrer <cody@instructure.com>
Landon Wilkins