Define the cost of a request as (user cpu + time spent in db queries),
then store that using a leaky bucket algorithm in redis. The algorithm
is slightly modified from a normal leaky bucket, see the comments in the
code.
The parameters (maximum, hwm, outflow) are configurable Settings.
Because this code path is hit on every request, I've tried to keep the
added latency to a minimum.
Uses lua scripting in redis to avoid the latency of an extra round trip
(two round trips with lua, one at the beginning of the request and one
at the end, vs at least three without lua).
closes CNVS-7008
test plan:
* Given the default params, you're not going to ever hit the throttling
limit without making multiple requests in parallel. Normally local dev
environments won't process parallel requests, so you'll want to tweak
the config params.
* Setting.set('request_throttle.hwm', '2')
* Setting.set('request_throttle.outflow', '0.1')
* Make canvas http requests as a logged in user, and verify that after a
few requests in quick succession, you get a 403 response. After
waiting a second for your bucket to fall back below the threshold, you
can make requests again.
* Do the same using the api and sending your access token in the query
string or http auth header.
* This should work correctly with a clustered redis as well.
* If possible it'd be good to test parallel requests as well. The code
should handle this correctly and combine the cost of the parallel
requests, rather than stomping on the values.
Change-Id: I1fdfd4e009d81bd6525bcf45a93437f4c395f129
Reviewed-on: https://gerrit.instructure.com/24256
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
We'd like to re-use some of this code elsewhere, and there's nothing
canvas-specific about it.
refs CNVS-7008
test plan: redis delayed job specs still pass
Change-Id: Iaa19ee60419d01bb451cccf2861a1015a7ba087a
Reviewed-on: https://gerrit.instructure.com/24255
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
We have a fix on master that is not yet in a release gem
refs CNVS-7945
test plan: notifications should still be successfully sent in all situations,
such as replying to discussions. this new version of the gem also fixes
an issue where <img> tags weren't getting replaced with their alt text
in some situations, which is an unrelated bug "fix". this doesn't appear
to affect canvas negatively in my testing.
Change-Id: Idbf52aa006d04252b7870cd0a3208d7bdc0e20e3
Reviewed-on: https://gerrit.instructure.com/24024
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
fixes CNVS-7379
test plan:
1. Log in as a student who has taken quizzes and has had assignments graded.
2. On the main and course pages, try to click on an Assignment/ Quiz link under Recent Feedback
3. The links should work without error
Change-Id: I669c057add5fc14df891608054a997b33939fd0c
Reviewed-on: https://gerrit.instructure.com/23242
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
some bugfixes, esp. around timezones and DST
Change-Id: I53df7f7e8d9af38dd300828795d017df7f304a57
test-plan: N/A
Reviewed-on: https://gerrit.instructure.com/23126
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
closes CNVS-7093
the changelog from rails 2.3.18 is at https://github.com/makandra/rails/commits/2-3-lts
so far it's mostly packaging changes, with explict config options to opt
out of some stuff like enable_xml_parsing
test plan:
* without RAILS_LTS file, specs should pass, general regression testing
* ditto with RAILS_LTS file pointing at the private gem repo (ask for
creds, we aren't allowed to share them publicly)
Change-Id: I3526c57ce5769dd73c84ffe8055fa084874b2bd5
Reviewed-on: https://gerrit.instructure.com/21654
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
fixes CNVS-6734
also displays errors encountered when joining
a conference rather than leaving it unhandled
testing steps:
* a legacy conference should work to start or join
after the update
* prior performing the update, create a conference
that is not yet started
* prior performing the update, create a conference
that has started but not yet ended
* verify both conferences can be started or joined
* new conferences should work to create and join
* conference unique ID clashes should error gracefully
* create a new conference, in a console, change the
setting :meeting_url_id to be set to an existing
meetings meeting_url_id and verify attempts to join
it fail with a nicer error message.
* in the console:
c = WebConference.find [insert ID of meeting]
c.settings[:meeting_url_id] = 'canvas-meeting-[ID of legacy meeting]'
c.save
* reload the conferences web page and attempt to join
the meeting
Change-Id: I39085eba54f1cfe51335d3df5c81df1930791a8e
Reviewed-on: https://gerrit.instructure.com/22196
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jon Willesen <jonw@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Mark Ericksen <marke@instructure.com>
refs gh-319
if we have pg_collkey, use the current locale instead of 'root', but
only if the current locale differs (so pg can still use the index
as much as possible)
test plan:
* add three users to a course, named "n", "na", and "ñ"
* set your language to English
* the people page of the course should sort as "n", "ñ",
"na"
* set your language to Spanish
* the people page of the course should sort as "n", "na",
"ñ"
Change-Id: Id01be92cb2dc103cc5f9e651da43cba6597bab22
Reviewed-on: https://gerrit.instructure.com/21648
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
This allows us to upgrade our routes file to rails 3 syntax while still
staying compatible with rails 2.3
refs CNVS-5146
test plan:
This affects literally every URL in Canvas, but shouldn't introduce any
new behavior. The automated integration and selenium tests still pass,
which is a good smoke test. Other than that, it's really just making
sure that routes are generated and recognized as before (regression
testing).
Change-Id: I443d006e3fcb5a0a0f8d6db46a8873a498ae7fd4
Reviewed-on: https://gerrit.instructure.com/21729
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
* disable i18nema until we're on rails 3.2
* fix how we access session_options
* fix how we access the basic auth header
* fix attachment_fu callbacks
test plan: none of these fixes should affect rails 2
Change-Id: I7f9b6f18c04d51284ec3e5e9fdd39fb93539c91b
Reviewed-on: https://gerrit.instructure.com/21728
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
test plan:
1. specs should pass
2. run canvas with RAILS_LOAD_ALL_LOCALES=1
3. you should be able to switch locales and see translations
4. run i18n:generate on the parent commit and on this commit, the
resulting config/locales/generated/en.yml should be identical
Change-Id: I5164943dc6e682c663145e91d7a9791331e9b4cc
Reviewed-on: https://gerrit.instructure.com/21014
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Marc LeGendre <marc@instructure.com>
Product-Review: Marc LeGendre <marc@instructure.com>
jammit can use sass to minify css. this is great
because now we don't need the yui-compressor.
which means that we don't need java.
Change-Id: I22d0f35340077a866ffe1000443bc5a061a9e613
Reviewed-on: https://gerrit.instructure.com/11588
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Bryan Madsen <bryan@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
test plan:
* create and join an adobe connect conference; verify
that it works as expected.
Change-Id: I30a1748e4c04da478bfbe8ddde2802efa90022df
Reviewed-on: https://gerrit.instructure.com/20512
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
Product-Review: Marc LeGendre <marc@instructure.com>
QA-Review: Marc LeGendre <marc@instructure.com>
fixes CNVS-5493
test steps:
Trigger the following emails and ensure that the HTML and plain text
versions of the email appear proper.
* conversation message received
* new announcement
* new discussion entry
* new discussion topic
* updated wiki page
* summary (daily or weekly)
Change-Id: I9638badb500eeb6cb23e4a83bd8395cd5dc40608
Reviewed-on: https://gerrit.instructure.com/20296
Reviewed-by: Jon Jensen <jon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Eric Berry <ericb@instructure.com>
QA-Review: Eric Berry <ericb@instructure.com>
- Pulls in latest fixes to LogOutRequest signatures
- Includes XML certificate in signed LogOutRequest
refs CNVS-5576
Test Plan:
- Set up shibboleth as per instructions in the wiki
- Using old code, log in as test user
- Watching SAML debugging, observe that Shibboleth responds to log out
message with error code
- Restart canvas with updated ruby-saml-mod gem
- Log in again as test user
- Verify that shibboleth accepts log out request signature
Change-Id: I1a700fc1c27738812b4ee6773500240c63d39735
Reviewed-on: https://gerrit.instructure.com/20444
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Reviewed-by: Paul Hinze <paulh@instructure.com>
Product-Review: Paul Hinze <paulh@instructure.com>
when a user explicitly logs out of one pseudonym session, invalidate all
the others
fixes CNVS-1923
test-plan:
- create a user in two different accounts
- log them in to both accounts
- click "log out" in one account
- should be logged out of both accounts
Change-Id: I79e70017d753c8201429901421e015f5d20e2000
Reviewed-on: https://gerrit.instructure.com/20096
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
The combination of these two changes fixes an issue with using extended
utf-8 chars in cql queries (including inserts)
fixes CNVS-5719
test plan: enable cassandra page views, and from a script/console run a
test query with extended utf-8 data, for instance:
PageView::EventStream.database.execute("SELECT * FROM page_views WHERE request_id = ?", "test \xEF\xBF\xBD one")
This will return no results since that request id isn't a UUID, but the
key is it shouldn't error.
Change-Id: I1dafb6165cdda65ab64267edbf9fabc4fccd783c
Reviewed-on: https://gerrit.instructure.com/20397
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
also remove the need for Guardfile within plugins (and stop including
them)
note that guard still can't detect changes to files in symlinked plugins
on the mac (due to fsevent), though you can hit enter to recompile everything
test plan:
1. clone a plugin into vendor/plugins
2. start up guard
3. edit a coffee file in the plugin
4. guard should detect it and compile it in the right place
5. edit a coffee spec file in the plugin
6. guard should detect it and compile it in the right place
7. edit a handlebars file in the plugin
8. guard should detect it and compile it in the right place
9. hit enter
10. guard should compile all coffee/handlebars files in the right place
Change-Id: I1e7c12f1368af66dee024e258899412526bb3fd2
Reviewed-on: https://gerrit.instructure.com/20219
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Ryan Florence <ryanf@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
This gem is a no-op in ruby 1.9, it only applies to 1.8.7
Change-Id: Ibea9808ea4981e581988e05dfde824dd8304dd8e
Reviewed-on: https://gerrit.instructure.com/20345
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
- update the Gemfile to be 1.9 only, and raise an exception on wrong
ruby version
- remove RUBY_VERSION checks, replacing with the applicable code
- remove the FasterCSV compatibility shim, just use CSV now
test plan: trying to bundle install on ruby 1.8 or 2.0 should raise an
exception, specs should pass, canvas should work as normal on 1.9
Change-Id: I49088e9d227c59c6d5d5acb417c2df971129474a
Reviewed-on: https://gerrit.instructure.com/19806
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
replaced the mailman gem with custom code with more error
handling. this will allow the incoming message processor to
continue processing messages after encountering a message with
an encoding or parsing error. the broken messages will be moved
aside to a separate folder for later inspection.
fixes CNVS-4970
test plan:
- read up on the new incoming_mail.yml configuration settings.
- configure incoming_mail.yml with the test imap accounts
using legacy settings and check for regressions.
- reconfigure incoming_mail.yml to read from a directory.
- copy some testing email files into the configured directory.
test files should be a mix of:
- emails with encoding errors
- emails with syntax errors
- normal emails
- all of the normal emails should be processed normally
- all of the error emails should be moved into the error
subdirectory
Change-Id: I0f946a56b41492f007db2775aa6da3cdfa4fdd3f
Reviewed-on: https://gerrit.instructure.com/19729
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
Product-Review: Marc LeGendre <marc@instructure.com>
QA-Review: Marc LeGendre <marc@instructure.com>
fixes CNVS-5367
this allows hairtrigger and our i18n extraction code to parse 1.9 syntax
correctly.
also fixed a issue with the i18n rake tasks, so symlinks in
vendor/plugins are now followed.
test plan: without this commit, run rake i18n:generate. then copy
config/locales/generated/en.yml somewhere.
then apply this commit, update your bundle, and run rake i18n:generate again.
compare the new en.yml against the saved one, they should be the same
(unless you have symlinks in vendor/plugins, in which case the new one
will have more strings but existing strings should be the same)
Change-Id: If0df5eae25c59822a9d3c2738fe9549a756ff9e9
Reviewed-on: https://gerrit.instructure.com/19693
Reviewed-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
extracted out of canvas-lms
refs CNVS-4713
test plan:
* actions that use a slave should still work (dashboard render)
* you should be able to switch envs and users in console
Change-Id: I07dda8057cf94383bc4579f1ef6b5a4b3ffc20b5
Reviewed-on: https://gerrit.instructure.com/19287
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
This will help immensely when debugging slow or misbehaving queries.
Thanks to 37signals: https://github.com/37signals/marginalia
closes CNVS-5108
test plan:
cp config/marginalia.yml.example config/marginalia.yml
then edit config/marginalia.yml to uncomment the production block and
change "production" to "development".
run Canvas, and see in your log file how SQL statements have some extra,
useful information in a comment at the end. Run delayed jobs, and you'll
also see useful info on SQL statements in the log -- including job tag,
and context_id in this case will be job id.
Change-Id: I7988a9afaf674bd00c3edc6adafccaf51e7ff60f
Reviewed-on: https://gerrit.instructure.com/16008
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
test plan:
1. start guard
2. save a file in app/stylesheets
- styleguide should be created (guard will
tell you)
3. hit enter
- styleguide should be created
Change-Id: I891a906602b6df0f964f2e502f124f5d05b4b796
Reviewed-on: https://gerrit.instructure.com/19525
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Ryan Florence <ryanf@instructure.com>
QA-Review: Ryan Florence <ryanf@instructure.com>
also added styles to make .form-controls look
good in dialogs.
closes #CNVS-4302
Change-Id: Ibe54ee4046ac255b0b0ea83d32afc88e4a820464
Reviewed-on: https://gerrit.instructure.com/19331
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
QA-Review: Ryan Florence <ryanf@instructure.com>
Product-Review: Ryan Florence <ryanf@instructure.com>
>= 0.11.0 passes all params on to libpq, so need to restrict our initializer
to restrict the params from database.yml to avoid errors
Change-Id: Ie7cab5163a960b05eb3feb4c8487f0a1a36ae4a1
Reviewed-on: https://gerrit.instructure.com/19370
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
delayed job workers segfault often when using libxml2 and ruby 1.9
these issues are not present when using version 2.9.0 of libxml2,
however, newer versions of nokogiri and libxml-ruby are required in
order to compile against the newer libxml2
fixes #CNVS-4669
Change-Id: I5100ba6e0f4779da49bce471cff03cb83b3c06b2
Reviewed-on: https://gerrit.instructure.com/18658
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
QA-Review: Adam Phillipps <adam@instructure.com>
Brian Palmer