refs CNVS-24699
test plan:
* go to /health_check.json
* it should include an installation_uuid
Change-Id: I4eedd51ccee7a5cdcd15df4af30443ff9fa01145
Reviewed-on: https://gerrit.instructure.com/66219
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
fixes CNVS-21362
test plan:
* make sure you have pg_collkey installed
* test user search
* it shouldn't asplode
Change-Id: I911d57ea950679b3e26390c140149c528eae0a2d
Reviewed-on: https://gerrit.instructure.com/56969
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
previously, when using `bin/spring`, often the error
`Expected <snip>/lib/canvas/draft_state_validations.rb to define
Canvas::DraftStateValidations` would come up when modifying application
code (not spec code). The only away around this problem would be to issues
a `bin/spring stop` command. The problem appears to be a Rails loading
convention that canvas wasn't respecting so this commit attempts to
correct this issue so that spring no longer blows up on a regular basis.
Change-Id: Icd44f4dceb75dc965f0c9a50b500f505c82cd15b
Fixes: CNVS-21552
Reviewed-on: https://gerrit.instructure.com/57319
Tested-by: Jenkins
Reviewed-by: Dylan Ross <dross@instructure.com>
Reviewed-by: Cameron Matheson <cameron@instructure.com>
Product-Review: Derek Bender <dbender@instructure.com>
QA-Review: Adrian Foong <afoong@instructure.com>
closes CNVS-6016
No more error reports! (soon)
this commit builds up sentry integration through the new
Canvas::Errors module, along with other things that need
to happen on every exception. ErrorReports
should now get pushed towards just being used for representing
a complaint a user filed via the get help form.
I fixed about half the things that got linted as well
while I was in here, but because this touches to much
I fear divergence from tackling too many (I think we
can safely say it's "better than we found it")
I left a lot of the infrastructure for error reports in place
until other commits for plugins can be merged
TEST PLAN:
1) setup your raven.yml config file with the dsn for our
sentry install
2) force an error to happen in a request response cycle.
3) see the error in sentry
4) force an error to happen in a job
5) see the error in sentry
6) statsd increments shoudl still fire
7) for the moment, an error report should still get created.
Change-Id: I5a9dc7214598f8d5083451fd15f0423f8f939034
Reviewed-on: https://gerrit.instructure.com/51621
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins
QA-Review: August Thornton <august@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
had to change the level that marshalling is disabled for the data
redis so that marshalling can be enabled for cache
test plan:
* configure redis.yml
* configure cache_store.yml for development/production to be
cache_store: redis_store and nothing else
* open console
* Canvas.redis.set('a', 1); Canvas.redis.get('a') should return
"1" (as a string)
* Rails.cache.write('a', 1); Rails.cache.read('a') should return
1 (as a Fixnum)
* Canvas.redis.__getobj__.object_id should be the same as
Rails.cache.instance_variable_get(:@data).object_id
* netstat -an should show only one connection from your console
process to the redis server
* with an already populated cache (having used Canvas before
applying this commit), click around Canvas, and make sure
nothing breaks
Change-Id: I507b95af3ec35f65501bb61e428a3e8479b8f895
Reviewed-on: https://gerrit.instructure.com/48134
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
QA-Review: August Thornton <august@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
fixes CNVS-15130
allows access to a plugin's models (during truncation for specs),
migrations, assets, periodic jobs, prepended routes, api routes (during
documentation), flagging as reloadable, spec docs, etc. once the plugin
lives in gems/plugins/ instead of vendor/plugins/
Change-Id: Ia7a2fdf5ad6258c5de962e982b6bf62ebaf4d075
Reviewed-on: https://gerrit.instructure.com/42287
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
fixes CNVS-15414
This applies to rails3 as well, we just use the appropriate version of
redis-rails for each.
test plan: redis should work as before, for both cache and data.
Change-Id: Ic34729190936a133b908e31ff186934fa435c1b6
Reviewed-on: https://gerrit.instructure.com/40898
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
test plan: with a VERSION file in your rails root, run delayed jobs, the
process name will include the contents of the VERSION file. with no
VERSION file, it won't.
Change-Id: I773ff4ace5facdb48d415ad7528daaf761aeab1d
Reviewed-on: https://gerrit.instructure.com/40162
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
(switchman handles the details)
test plan:
* in cache_store.yml, have something like:
oneoff:
cache_store: redis_store
development: oneoff
* launch rails console in development mode; Rails.cache should be a redis store
Change-Id: Ie12e655264b1f8c4a5df6ef31cf84d12cc913eb6
Reviewed-on: https://gerrit.instructure.com/39923
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
test plan:
- add a `timeout: 1` option to redis.yml
- confirm that rails cache and redis operations complete as normal
- to test timeout, you can set up a dummy tcp server that listens but
never accepts, add it to the redis config, and verify that the (first)
redis operation times out as expected. after the first, it'll be
blacklisted for 5m as usual.
Change-Id: I2494b4b319afffcf1ea6ab87eb1c3d7d3baec58b
Reviewed-on: https://gerrit.instructure.com/39895
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
closes CNVS-13444
test plan:
- specs should pass
- external services (like turnitin or crocodoc) should still work
Change-Id: I223320f8c6b0dafd79d6e47fbdaedfb6300dc512
Reviewed-on: https://gerrit.instructure.com/35939
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Amber Taniuchi <amber@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
refs CNVS-13024
Setting wasn't properly being initialized as unsharded because
it was loading before Switchman. The reason we need Setting before
switchman is just for yaml loading, so split that into its own
class.
Change-Id: I5456e103cb216dba2d5af4e9c20a697b468c923b
Reviewed-on: https://gerrit.instructure.com/35043
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
test-plan:
- create a cache_store.yml that's not a hash
- try to boot canvas, get an appropriate error
- create a cache_store.yml with non-hash values for the top level keys
(e.g. entire file is "cache_store: redis_store")
- try to boot canvas, get an appropriate error
Change-Id: I4e8965a61c11c64e1d81894e8ae316761ac12f1b
Reviewed-on: https://gerrit.instructure.com/29663
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
test-plan:
- don't have a config/cache_store.yml
- try and boot canvas
Change-Id: I58ceca7bc3b618421e7d66987bbd18b2646babf4
Reviewed-on: https://gerrit.instructure.com/29582
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: James Williams <jamesw@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
refs CNVS-9546
for rails2, when using multiple environment-conditioned caches,
middleware must be inserted for each cache. so we need to instantiate
them and manage the middleware at boot. adjust accordingly
for rails3, we're using switchman and switchman has been updated to
manage this for us. we just need to give it the full cache configuration
map up front.
also changes the default for Canvas.cache_store_config's second
parameter to true, extracting the nil_is_nil=false behavior into the
fallback value for Rails.env in the new Canvas.cache_stores method. the
only remaining callers of this method are in some plugins which passed
an explicit true value for that parameter. once those plugins are
updated to access Canvas.cache_stores directly,
Canvas.cache_store_config will be removed.
Change-Id: I80eb09679303f2778a6ddc9ab476e695a6754465
Reviewed-on: https://gerrit.instructure.com/28441
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
refs CNVS-4704
we can use the built in :null_store with rails3
Change-Id: I0dc6c441da4f300fe8ee870b8c676c8e449d1830
Reviewed-on: https://gerrit.instructure.com/26332
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
avoids warnings about DYLD_ env vars if you have those set
Change-Id: If3a5ea88727a64fa61dd7591a82f445b7491b042
Reviewed-on: https://gerrit.instructure.com/25833
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
now that we have SIGHUP, we were changing everything to it anyway,
so just let caching in-proc be the default
Change-Id: Id1b44722522ac9693b17695da7107c99a359d5ac
Reviewed-on: https://gerrit.instructure.com/25020
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
refs CNVS-8371
test plan:
* trigger 3 ldap timeouts, see test plan in c17f3570bd
* you'll be blocked from further ldap attempts for 1 minute. after that
minute passes, verify that canvas starts making ldap attempts again.
Change-Id: I8ac6b74bd75f6f3e18deaf3a1fabd01fb867c26e
Reviewed-on: https://gerrit.instructure.com/25047
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Brian Palmer <brianp@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
The previous code was a one-off written before timeout_protection
existed. This means that we'll now allow N timeouts (3 by default) in
the given period, rather than refusing to connect to the LDAP server
after just one timeout.
closes CNVS-8371
test plan:
* Configure an account to use ldap. Rather than setting up a real ldap
server, it's sufficient for this testing to just use nc or another
application to listen on the port you specify in the account config.
* Attempt to login to the account, and see canvas in your nc output.
Allow it to timeout. Attempt again, and canvas will hit your "ldap
server" again. After 3 timed out attempts, canvas will blacklist your
server for 1 minute.
* Also verify that logging in with ldap still works against a real ldap.
Change-Id: I60293d01690be3cc24f57b8bcd5c6c52e23fc2a9
Reviewed-on: https://gerrit.instructure.com/24657
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
incoming message processor should not wait indefinitely on
external connections. it now implements timeouts on external
connections that are controlled by the Canvas.timeout_protection
method. It uses incoming_message_processor as the serivce name,
which makes these meaningful settings to change the timeout
behavior:
- service_incoming_message_processor_timeout
- service_incoming_message_processor_cutoff
- service_incoming_message_processor_error_ttl
fixes CNVS-8198
test plan:
- see the incoming message processor wiki page for instructions
on how to do the configuration and run these tests.
- create a fifo in an empty directory, configure
incoming_mail.yml to read from the directory, run
script/process_incoming_emails and make sure it exits in
30 seconds or so.
- use nc -l <port> to listen on a port, configure
incoming_mail.yml to access that port on localhost, and run
script/process_incoming_emails and make sure it exits in
30 seconds or so.
- run script/process_incoming_emails against an actual email
account and make sure it can still process emails normally.
Change-Id: I23c67c1e8c0581a1e6ca69ab2c7b8855090688d1
Reviewed-on: https://gerrit.instructure.com/24483
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Reviewed-by: Jon Willesen <jonw@instructure.com>
Product-Review: Jon Willesen <jonw@instructure.com>
test plan:
curl -H 'Accept: application/json' http://<domain>/health_check
If you have a VERSION file in your canvas dir it'll report that
version, otherwise "Unknown"
Change-Id: I43c72140541ea441d4ec96a4c179d8db775843b7
Reviewed-on: https://gerrit.instructure.com/24595
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
This data will be needed for throttling abusive users based on cpu + db
usage.
refs CNVS-7008
This also replaces Canvas.sample_cpu_time with just using Process.times,
which works cross-platform and is likely faster.
test plan: specs, and you can verify that the STAT line is still being
logged to the rails log and the statsd messages are still being sent
over UDP if statsd is enabled
Change-Id: I6a5aac389e6de9caa12751d57532f32464acaeca
Reviewed-on: https://gerrit.instructure.com/23499
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
Unfortuanately the current ruby redis gem doesn't support pipelining
with a redis cluster, even if all the commands are acting on the same
key. Since we're working on turning the data redis into a cluster, these
have got to go.
test plan: specs should cover all this. could also do a quick regression
test on failed logouts (shouldn't error, should still limit failed login
attempts)
Change-Id: Ifa950268416e9086b72610914d7efc8a816d628b
Reviewed-on: https://gerrit.instructure.com/23457
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
This can successfully load rails console and rails server. There are
many, many problems still. The idea is this won't change anything under
rails 2.3, it's all backwards compatible.
closes CNVS-4711
test plan: `touch RAILS3` in your Canvas Rails.root directory. The run
`bundle update` and verify that you get rails 3 installed. Run `bundle
exec rails c` to load console or `bundle exec rails s` to start a
webrick server. You can login, though the dashboard currently breaks.
Also jammit isn't working yet.
But more importantly, Rails 2.3 should still work same as ever. All
tests should pass, and a basic regression sanity check would be good too.
Change-Id: Idd6f35de88adde84cd2db3a650f44b71bd6e9684
Reviewed-on: https://gerrit.instructure.com/18453
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
allows it to be a different shard than where the file is. I had
to remove type casting from dynamic finders that don't know how
to deal with non-integral global ids.
also cache s3 urls on the same shard as the attachment
test plan:
* have multiple shards and S3 storage
* have a single safe files domain
* you should be able to upload and download files in all shards
* verify that it's going against the files domain, not the normal
domain
Change-Id: I2b498fc1df20d5b43bf20f702580451621eeaf6a
Reviewed-on: https://gerrit.instructure.com/15158
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
This supports all the features of our current delayed jobs backend,
including priorities, queues, jobs scheduled for the future, strands,
periodic jobs, etc.
Atomicity is guaranteed by using the new Lua scripting features of Redis
2.6, so this requires 2.6 (which is still only a release candidate as of this
time).
It's not yet production ready (see the TODO at the top of the
redis/job.rb file), so trying to start canvas in prod with redis jobs
configured will raise an error as a safeguard.
The tests have been modified to all pass with redis configured as the
jobs backend. Some selenium specs were removed because the jobs UI lost
some functionality that I didn't think was important enough to port to
redis (listing the most popular tags for future and failed jobs, for
example).
Change-Id: Ie57b15bae1d4ba7b2b2344c872411165551d1ac8
Reviewed-on: https://gerrit.instructure.com/12120
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
no user visible change
test plan:
* smoke test with and without caching configured
Change-Id: Ia21a996988021d647e56f85cd8ce818b64001681
Reviewed-on: https://gerrit.instructure.com/13248
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Canvas will use this to pull the data about a link before creating a
collection item (currently uses embed.ly)
test plan: not possible to test this through the UI yet
Change-Id: Ie248be4081871aa3aa747510d96edc3c7cc3a0a6
Reviewed-on: https://gerrit.instructure.com/10777
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Ryan Shaw <ryan@instructure.com>
test plan: run canvas through passenger and give it a bad IP for the
redis box, verify it can spin up new processes
Change-Id: I7d847da1c290261f89cbf2d792c875eefd20fe2c
Reviewed-on: https://gerrit.instructure.com/10551
Reviewed-by: Zach Wily <zach@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
and display the revision on /plugins
test plan: without a VERSION file, visit /plugins and see the "Unknown"
displayed. With a VERSION file, visit /plugins and see the first line of
that file displayed.
Change-Id: I0bcb3c855c70ba444158e17302eadab317dffb02
Reviewed-on: https://gerrit.instructure.com/9197
Reviewed-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
test plan: enable the plugin in /plugins, enter your creds, and start a
wiziq conference. teachers should be able to launch the conference, and
multiple students should be able to join.
closes#5738
Change-Id: I609542a0b13271e85dc18ff8b82ad6714736cc72
Reviewed-on: https://gerrit.instructure.com/8160
Tested-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Brian Whitmer <brian@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
The current version of redis-cache takes only the list of servers, it
doesn't support other config options like key prefix or database number.
Change-Id: If233c46d77b58650627dda75d7a8e1f2f212eb7f
Reviewed-on: https://gerrit.instructure.com/4801
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Right now this is just a data dump, pretty much.
Change-Id: I401cef17e2129556d99ca0a6547ae52712915650
Reviewed-on: https://gerrit.instructure.com/3970
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
Cody Cutrer