refs CNVS-13024
Setting wasn't properly being initialized as unsharded because
it was loading before Switchman. The reason we need Setting before
switchman is just for yaml loading, so split that into its own
class.
Change-Id: I5456e103cb216dba2d5af4e9c20a697b468c923b
Reviewed-on: https://gerrit.instructure.com/35043
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
test-plan:
- create a cache_store.yml that's not a hash
- try to boot canvas, get an appropriate error
- create a cache_store.yml with non-hash values for the top level keys
(e.g. entire file is "cache_store: redis_store")
- try to boot canvas, get an appropriate error
Change-Id: I4e8965a61c11c64e1d81894e8ae316761ac12f1b
Reviewed-on: https://gerrit.instructure.com/29663
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
test-plan:
- don't have a config/cache_store.yml
- try and boot canvas
Change-Id: I58ceca7bc3b618421e7d66987bbd18b2646babf4
Reviewed-on: https://gerrit.instructure.com/29582
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: James Williams <jamesw@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
refs CNVS-9546
for rails2, when using multiple environment-conditioned caches,
middleware must be inserted for each cache. so we need to instantiate
them and manage the middleware at boot. adjust accordingly
for rails3, we're using switchman and switchman has been updated to
manage this for us. we just need to give it the full cache configuration
map up front.
also changes the default for Canvas.cache_store_config's second
parameter to true, extracting the nil_is_nil=false behavior into the
fallback value for Rails.env in the new Canvas.cache_stores method. the
only remaining callers of this method are in some plugins which passed
an explicit true value for that parameter. once those plugins are
updated to access Canvas.cache_stores directly,
Canvas.cache_store_config will be removed.
Change-Id: I80eb09679303f2778a6ddc9ab476e695a6754465
Reviewed-on: https://gerrit.instructure.com/28441
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
refs CNVS-4704
we can use the built in :null_store with rails3
Change-Id: I0dc6c441da4f300fe8ee870b8c676c8e449d1830
Reviewed-on: https://gerrit.instructure.com/26332
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
avoids warnings about DYLD_ env vars if you have those set
Change-Id: If3a5ea88727a64fa61dd7591a82f445b7491b042
Reviewed-on: https://gerrit.instructure.com/25833
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
now that we have SIGHUP, we were changing everything to it anyway,
so just let caching in-proc be the default
Change-Id: Id1b44722522ac9693b17695da7107c99a359d5ac
Reviewed-on: https://gerrit.instructure.com/25020
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
refs CNVS-8371
test plan:
* trigger 3 ldap timeouts, see test plan in c17f3570bd
* you'll be blocked from further ldap attempts for 1 minute. after that
minute passes, verify that canvas starts making ldap attempts again.
Change-Id: I8ac6b74bd75f6f3e18deaf3a1fabd01fb867c26e
Reviewed-on: https://gerrit.instructure.com/25047
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Brian Palmer <brianp@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
The previous code was a one-off written before timeout_protection
existed. This means that we'll now allow N timeouts (3 by default) in
the given period, rather than refusing to connect to the LDAP server
after just one timeout.
closes CNVS-8371
test plan:
* Configure an account to use ldap. Rather than setting up a real ldap
server, it's sufficient for this testing to just use nc or another
application to listen on the port you specify in the account config.
* Attempt to login to the account, and see canvas in your nc output.
Allow it to timeout. Attempt again, and canvas will hit your "ldap
server" again. After 3 timed out attempts, canvas will blacklist your
server for 1 minute.
* Also verify that logging in with ldap still works against a real ldap.
Change-Id: I60293d01690be3cc24f57b8bcd5c6c52e23fc2a9
Reviewed-on: https://gerrit.instructure.com/24657
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
incoming message processor should not wait indefinitely on
external connections. it now implements timeouts on external
connections that are controlled by the Canvas.timeout_protection
method. It uses incoming_message_processor as the serivce name,
which makes these meaningful settings to change the timeout
behavior:
- service_incoming_message_processor_timeout
- service_incoming_message_processor_cutoff
- service_incoming_message_processor_error_ttl
fixes CNVS-8198
test plan:
- see the incoming message processor wiki page for instructions
on how to do the configuration and run these tests.
- create a fifo in an empty directory, configure
incoming_mail.yml to read from the directory, run
script/process_incoming_emails and make sure it exits in
30 seconds or so.
- use nc -l <port> to listen on a port, configure
incoming_mail.yml to access that port on localhost, and run
script/process_incoming_emails and make sure it exits in
30 seconds or so.
- run script/process_incoming_emails against an actual email
account and make sure it can still process emails normally.
Change-Id: I23c67c1e8c0581a1e6ca69ab2c7b8855090688d1
Reviewed-on: https://gerrit.instructure.com/24483
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Reviewed-by: Jon Willesen <jonw@instructure.com>
Product-Review: Jon Willesen <jonw@instructure.com>
test plan:
curl -H 'Accept: application/json' http://<domain>/health_check
If you have a VERSION file in your canvas dir it'll report that
version, otherwise "Unknown"
Change-Id: I43c72140541ea441d4ec96a4c179d8db775843b7
Reviewed-on: https://gerrit.instructure.com/24595
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
This data will be needed for throttling abusive users based on cpu + db
usage.
refs CNVS-7008
This also replaces Canvas.sample_cpu_time with just using Process.times,
which works cross-platform and is likely faster.
test plan: specs, and you can verify that the STAT line is still being
logged to the rails log and the statsd messages are still being sent
over UDP if statsd is enabled
Change-Id: I6a5aac389e6de9caa12751d57532f32464acaeca
Reviewed-on: https://gerrit.instructure.com/23499
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
Unfortuanately the current ruby redis gem doesn't support pipelining
with a redis cluster, even if all the commands are acting on the same
key. Since we're working on turning the data redis into a cluster, these
have got to go.
test plan: specs should cover all this. could also do a quick regression
test on failed logouts (shouldn't error, should still limit failed login
attempts)
Change-Id: Ifa950268416e9086b72610914d7efc8a816d628b
Reviewed-on: https://gerrit.instructure.com/23457
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
This can successfully load rails console and rails server. There are
many, many problems still. The idea is this won't change anything under
rails 2.3, it's all backwards compatible.
closes CNVS-4711
test plan: `touch RAILS3` in your Canvas Rails.root directory. The run
`bundle update` and verify that you get rails 3 installed. Run `bundle
exec rails c` to load console or `bundle exec rails s` to start a
webrick server. You can login, though the dashboard currently breaks.
Also jammit isn't working yet.
But more importantly, Rails 2.3 should still work same as ever. All
tests should pass, and a basic regression sanity check would be good too.
Change-Id: Idd6f35de88adde84cd2db3a650f44b71bd6e9684
Reviewed-on: https://gerrit.instructure.com/18453
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
allows it to be a different shard than where the file is. I had
to remove type casting from dynamic finders that don't know how
to deal with non-integral global ids.
also cache s3 urls on the same shard as the attachment
test plan:
* have multiple shards and S3 storage
* have a single safe files domain
* you should be able to upload and download files in all shards
* verify that it's going against the files domain, not the normal
domain
Change-Id: I2b498fc1df20d5b43bf20f702580451621eeaf6a
Reviewed-on: https://gerrit.instructure.com/15158
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
This supports all the features of our current delayed jobs backend,
including priorities, queues, jobs scheduled for the future, strands,
periodic jobs, etc.
Atomicity is guaranteed by using the new Lua scripting features of Redis
2.6, so this requires 2.6 (which is still only a release candidate as of this
time).
It's not yet production ready (see the TODO at the top of the
redis/job.rb file), so trying to start canvas in prod with redis jobs
configured will raise an error as a safeguard.
The tests have been modified to all pass with redis configured as the
jobs backend. Some selenium specs were removed because the jobs UI lost
some functionality that I didn't think was important enough to port to
redis (listing the most popular tags for future and failed jobs, for
example).
Change-Id: Ie57b15bae1d4ba7b2b2344c872411165551d1ac8
Reviewed-on: https://gerrit.instructure.com/12120
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
no user visible change
test plan:
* smoke test with and without caching configured
Change-Id: Ia21a996988021d647e56f85cd8ce818b64001681
Reviewed-on: https://gerrit.instructure.com/13248
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Canvas will use this to pull the data about a link before creating a
collection item (currently uses embed.ly)
test plan: not possible to test this through the UI yet
Change-Id: Ie248be4081871aa3aa747510d96edc3c7cc3a0a6
Reviewed-on: https://gerrit.instructure.com/10777
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Ryan Shaw <ryan@instructure.com>
test plan: run canvas through passenger and give it a bad IP for the
redis box, verify it can spin up new processes
Change-Id: I7d847da1c290261f89cbf2d792c875eefd20fe2c
Reviewed-on: https://gerrit.instructure.com/10551
Reviewed-by: Zach Wily <zach@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
and display the revision on /plugins
test plan: without a VERSION file, visit /plugins and see the "Unknown"
displayed. With a VERSION file, visit /plugins and see the first line of
that file displayed.
Change-Id: I0bcb3c855c70ba444158e17302eadab317dffb02
Reviewed-on: https://gerrit.instructure.com/9197
Reviewed-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
test plan: enable the plugin in /plugins, enter your creds, and start a
wiziq conference. teachers should be able to launch the conference, and
multiple students should be able to join.
closes#5738
Change-Id: I609542a0b13271e85dc18ff8b82ad6714736cc72
Reviewed-on: https://gerrit.instructure.com/8160
Tested-by: Brian Palmer <brianp@instructure.com>
Reviewed-by: Brian Whitmer <brian@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
The current version of redis-cache takes only the list of servers, it
doesn't support other config options like key prefix or database number.
Change-Id: If233c46d77b58650627dda75d7a8e1f2f212eb7f
Reviewed-on: https://gerrit.instructure.com/4801
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Right now this is just a data dump, pretty much.
Change-Id: I401cef17e2129556d99ca0a6547ae52712915650
Reviewed-on: https://gerrit.instructure.com/3970
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
in production mode we now cast to the appropriate type and issue a warning
if it can't be cleanly cast (e.g. '' -> 0). if all arguments are nil (or
empty arrays, e.g. find_by_id([])) and we aren't in a scope, issue a
warning (sometimes we really do want nil when we're in a scope, e.g. line
216 of app/models/folder/rb).
in development/test mode, we now raise errors in the two warning scenarios
above (though that is configurable).
fixed several places in the code where specs failed due to the change, or
where inputs to dynamic finders looked problematic
Change-Id: Ifea851cb14d3e89b6df08ade8e83934579678f8b
Reviewed-on: https://gerrit.instructure.com/3434
Reviewed-by: Zach Wily <zach@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
A background periodic process pulls them off the queue, consolidates
updates, and writes them to the database.
Change-Id: I854b57932e3f6e311ce8c67971ed5208034ca6a5
Reviewed-on: https://gerrit.instructure.com/2803
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Whitmer <brian@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Cody Cutrer