Commit Graph

1218 Commits

Author SHA1 Message Date
Cody Cutrer acd56b976b update mime-types
fixes gh-1390

Change-Id: Idab71c6e9cafd51bf6858f4ba1a07b38f934140e
Reviewed-on: https://gerrit.instructure.com/182668
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2019-02-27 17:05:14 +00:00
Cody Cutrer 15d13bff49 rip out ruby-saml-mod gem
Change-Id: Id6c71716a3060747b817e7c031487746aa170cc7
Reviewed-on: https://gerrit.instructure.com/177869
Tested-by: Jenkins
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
2019-02-22 19:20:32 +00:00
Cody Cutrer f1812f2299 add direct send_in_region helper
refs CORE-2438

Change-Id: I41bff34847009029fa631ff776cd99a9189846ac
Reviewed-on: https://gerrit.instructure.com/181239
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2019-02-22 19:09:15 +00:00
Cody Cutrer 8303272494 fix error thrown by marginalia if cached schema is wrong
basically, avoid trying to do column lookups at this point

Change-Id: Ib652d20cf3822ba6167a5687a5b14aadfbfc8427
Reviewed-on: https://gerrit.instructure.com/181242
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2019-02-22 18:14:48 +00:00
Cody Cutrer 2a5965c9be fix spec time limits
and no time limits while debugging

I'm guessing this is just older rubies being more lenient, and ruby
2.6 being more strict

Change-Id: I6dba7f4fca1e6129510f873016e0b72dfe550a72
Reviewed-on: https://gerrit.instructure.com/181240
Tested-by: Jenkins
Reviewed-by: James Williams <jamesw@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2019-02-13 19:33:52 +00:00
wdransfield 3400a1e43f Add scores MIME type
refs PLAT-4154

Test Plan:
Verify you can post results to the AGS using the
application/vnd.ims.lis.v1.score+json content-type

Change-Id: I2de0da013a93669b597aaac02070d2dac7ff207b
Reviewed-on: https://gerrit.instructure.com/178409
Tested-by: Jenkins
Reviewed-by: Marc Phillips <mphillips@instructure.com>
QA-Review: Marc Phillips <mphillips@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
2019-01-18 14:44:22 +00:00
wdransfield e46413fc72 Add custom AGS mime type
Closes PLAT-4154

Test Plan:
Attempt to use the AGS with the custom mime type. Verify line items can
be created.

Change-Id: Ib771ca04a0fbcda87bc7b169abe9fcafb11f4e47
Reviewed-on: https://gerrit.instructure.com/178334
Tested-by: Jenkins
Product-Review: Weston Dransfield <wdransfield@instructure.com>
Reviewed-by: Marc Phillips <mphillips@instructure.com>
QA-Review: Marc Phillips <mphillips@instructure.com>
2019-01-17 22:55:58 +00:00
Marc Phillips ac56ee8b64 Fix lti key rotation job
Missed fixing the namespace inside the block being used in the
delayed job.

refs PLAT-4048

Test Plan:
n/a

Change-Id: I0c6e38ac2074e546206eaecbf82fc2a33ba4963f
Reviewed-on: https://gerrit.instructure.com/176552
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Jenkins
Product-Review: Marc Phillips <mphillips@instructure.com>
2019-01-02 18:25:43 +00:00
James Williams 3e161cb6e8 partition messages
test plan:
* run the migrations
* sending messages and viewing them as an admin
 should work as before

closes #COMMS-1327

Change-Id: If955b49d07fa68f8a5151984eb4d9c3b97b118ee
Reviewed-on: https://gerrit.instructure.com/170657
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
2018-12-10 13:42:35 +00:00
Marc Phillips a9f0f83895 Fix Lti::Keystorage delayed job
Capitalization and snake-casing matter.

closes PLAT-3911

Test Plan:
 n/a

Change-Id: I2be342d57d12b89e491aa9ecd4b3423a35cfee38
Reviewed-on: https://gerrit.instructure.com/173989
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Tested-by: Jenkins
Product-Review: Marc Phillips <mphillips@instructure.com>
2018-12-03 18:10:40 +00:00
Cameron Matheson 39d831f6b5 graphql: ensure nodes field is always enabled
The graphql-ruby gem defaults to not including the nodes field for
connections defined with the legacy api

Test plan:
  All connections should now have a `nodes` field.

closes GQL-19

Change-Id: I5981b68cdbde73cac2ca06ca4ac90448ab348008
Reviewed-on: https://gerrit.instructure.com/173412
Tested-by: Jenkins
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
QA-Review: Cameron Matheson <cameron@instructure.com>
Product-Review: Cameron Matheson <cameron@instructure.com>
2018-11-27 19:36:30 +00:00
James Williams 6688ebefe9 don't return a next page header on last page when untotalled
closes #CORE-1770

Change-Id: Ic0907e2c6d7e01158ea9ad651bc676e1732aab73
Reviewed-on: https://gerrit.instructure.com/167143
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: James Williams <jamesw@instructure.com>
Product-Review: James Williams <jamesw@instructure.com>
2018-10-11 12:13:39 +00:00
Simon Williams b73b213908 correctly handle send_in_each_region if region config is not set
if not database server has a region set, the job still needs to run
once.

fixes gh-1344

test plan:
- on open source canvas, make sure incoming mail jobs are actually
queued and run

Change-Id: Id1cdd0da52b2a10356c2542e37b358278af2e697
Reviewed-on: https://gerrit.instructure.com/164657
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Steven Burnett <sburnett@instructure.com>
Product-Review: Steven Burnett <sburnett@instructure.com>
2018-09-20 13:17:55 +00:00
Cody Cutrer 40d18a74b8 fix conditional for faux-singleton periodic jobs
Change-Id: Icc743234201b8234052693dad2d45bb7f6c6e645
Reviewed-on: https://gerrit.instructure.com/164217
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-09-12 18:51:18 +00:00
Cody Cutrer 2aa4ff57d6 tweak jobs statsd a bit
Change-Id: If8da9d13963965b260438f07c68e9427a33508ee
Reviewed-on: https://gerrit.instructure.com/164179
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-09-12 16:46:28 +00:00
Cody Cutrer 76d8fd160f faux singleton per-shard periodic jobs
it's still on a strand per-db, but don't enqueue multiple per-shard

Change-Id: I32185d6fddd6007ef1dbc507e88aa10556ec4995
Reviewed-on: https://gerrit.instructure.com/164172
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-09-12 16:05:52 +00:00
Cody Cutrer 481d7bcedb lower the priority of all the per-shard periodic jobs
Change-Id: I14552221c9af399d6ca677ba4e7fc323bad38f7c
Reviewed-on: https://gerrit.instructure.com/164149
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-09-12 16:05:40 +00:00
Cody Cutrer a88865c149 bump switchman
and remove superfluous override that switchman handles

Change-Id: I3a512864f67d2f4a50de9b1d78a7cca10ec76419
Reviewed-on: https://gerrit.instructure.com/163625
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-09-07 20:41:10 +00:00
Cody Cutrer e2067794ce only send strand stats to statsd for selected strands
Change-Id: I5e2e3fe68c1b6c5578aa2f139d0f65fb8f152f50
Reviewed-on: https://gerrit.instructure.com/163020
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-09-06 15:45:17 +00:00
James Williams 776f42b988 stop and requeue materialized view jobs when slave is backed up
closes #CORE-1830

Change-Id: Ib40cf5c3a500cba5cc75b5826769cbd70cb24612
Reviewed-on: https://gerrit.instructure.com/163223
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2018-09-06 12:08:24 +00:00
Cody Cutrer ecc16d729a fold Setting.get_or_set into get with an option
so that it can use the same caching infra

Change-Id: Ie797a8f83d16def504b76003b472b538d4b52df5
Reviewed-on: https://gerrit.instructure.com/162504
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-29 19:02:57 +00:00
Cody Cutrer f5cc9f0a3b partition stats by strand as well as tag
refs GOOF-543

since some strand:tag isn't necessarily 1:1

Change-Id: I2a149c9b506379dbf2af1ac82f839e2704752af6
Reviewed-on: https://gerrit.instructure.com/162518
Tested-by: Jenkins
Reviewed-by: Josh Orr <jgorr@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-29 18:06:45 +00:00
Cody Cutrer 087240adb4 bump inst-jobs, switchman-inst-jobs
and log autoscaling actions even when autoscaling is enabled

Change-Id: I9f8c5c526707ebf2f3c085a0ce98d4b6ad30b414
Reviewed-on: https://gerrit.instructure.com/161806
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-28 18:56:46 +00:00
Cody Cutrer b60e4d7348 don't allow configuring saml_max_message_size
fixes CORE-1784

we've never adjusted it, and it's causing a thundering herd of connections
on HUP and restart

Change-Id: I2d479be48cb19de6389f03cf25e41154bb12bef6
Reviewed-on: https://gerrit.instructure.com/162333
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-28 18:46:27 +00:00
Cody Cutrer 45f7288fcf don't use COPY for find_in_batches during migrations
fixes CNVS-43369

it can cause the connection to swap, which confuses AR's advisory lock
protecting against concurrent migraitons

Change-Id: I0e75ffc76dbe7408221c9865b93d0561bcd46b5b
Reviewed-on: https://gerrit.instructure.com/161494
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Reviewed-by: Derek Bender <djbender@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-23 19:14:08 +00:00
Cody Cutrer b5e465ccd6 fix creating new shards from scratch
Change-Id: I95b1077ecf0bf8690e34b7664bb5b8c5060c71fa
Reviewed-on: https://gerrit.instructure.com/160991
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-15 23:33:46 +00:00
James Williams f6ce48e161 skip column cache in unusual cases
just in case we're trying to log a redis error. blowing the stack
isn't cool

Change-Id: I396584bc45f36089f2219ee5215745fcf6444ddc
Reviewed-on: https://gerrit.instructure.com/160951
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-15 21:31:08 +00:00
Cody Cutrer 29cab0d677 multicache and invalidate column definitions
closes CORE-1724

Change-Id: I2b57cf9c20b6c1dddc2fdd7f1c383ca3b56a3adb
Reviewed-on: https://gerrit.instructure.com/160907
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-15 18:25:56 +00:00
James Williams bfa174d7b0 rails 5.2: bump rails
woo

Change-Id: I1c3045f001626bee41ee669e8bbe86ea708b8080
Reviewed-on: https://gerrit.instructure.com/160230
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2018-08-13 15:18:04 +00:00
Cody Cutrer e8f0be2f5e rails 5.2: enforce raw sql for order by
Change-Id: I6667c7917306a8c67802b99f9a6ecbe758a0a146
Reviewed-on: https://gerrit.instructure.com/159913
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-08-08 17:31:14 +00:00
James Williams 6109d6e72e queue imp on a per-region basis
Change-Id: Ia1c3b06f8adffb822415b672912b873dd748197f
Reviewed-on: https://gerrit.instructure.com/158589
Reviewed-by: Steven Burnett <sburnett@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2018-08-01 15:21:21 +00:00
Cody Cutrer dfd60138ae Rails 5.2: Arel.sql
Change-Id: I62b5054d46148679bbf650e2cac495df6c37d8d4
Reviewed-on: https://gerrit.instructure.com/158979
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-07-30 19:13:06 +00:00
Han Ngo 27d24538d6 Run Key Rotation Script as a Periodic Job in Canvas
Closes PLAT-3508 & PLAT-3509

Test Plan:
- Verify the key rotate correctly (set current to past, future
to current, and the new key to future) in both cases: when there
is no existing key AND when there is key already in Consul
- Verify the script will be running every month
- Verify the keys got rotated using Imperium gem version '0.4.0'

Change-Id: I630f230b3cd1c515ebb266b532901b4260622173
Reviewed-on: https://gerrit.instructure.com/154529
Reviewed-by: Stewie aka Nicholas Stewart <nstewart@instructure.com>
QA-Review: Nathan Mills <nathanm@instructure.com>
Product-Review: Nathan Mills <nathanm@instructure.com>
Tested-by: Jenkins
2018-07-12 19:57:40 +00:00
Cody Cutrer 2971894311 rails 5.2: several Arel.sqls in AR::Base extensions
Change-Id: Ie8b4ba279376b0f9cf25f1e473bd29c3094a4137
Reviewed-on: https://gerrit.instructure.com/156641
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-07-10 15:16:34 +00:00
Cody Cutrer 5fd001935c register pg_lsn type
so that we don't get annoying warnings about assuming it's a string

Change-Id: I14e5741725247fb4937cda0a5e39366fb2086f12
Reviewed-on: https://gerrit.instructure.com/156609
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-07-09 20:25:15 +00:00
Cody Cutrer ce9fcf595f run counts reports in parallel
refs CORE-1505

also, don't build the overview report anymore (which requires summing up
all of the parts)

Change-Id: Icb4c317e65237cb4ea9a800d7742f36ebac79022
Reviewed-on: https://gerrit.instructure.com/154342
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-06-21 14:44:22 +00:00
Matt Sessions c97d6a5887 Fix assignment missing periodic job
Refs MBL-10482

Test Plan:
 - Create an observer alert threshold for
   assignment_missing between an observer and student
 - Create an assignment that is due in 1-2 minutes
 - Wait at least 5 minutes
 - Hit /api/v1/users/self/observer_alerts
 - An assignment_missing alert should exist

Change-Id: I7936d1eb7bcf347a776c3d0af675e61f5c25db98
Reviewed-on: https://gerrit.instructure.com/153446
Reviewed-by: Nate Armstrong <narmstrong@instructure.com>
QA-Review: Nate Armstrong <narmstrong@instructure.com>
Tested-by: Jenkins
Product-Review: Matthew Sessions <msessions@instructure.com>
2018-06-13 18:46:18 +00:00
Cody Cutrer 568fefc89e match job name to method name
Change-Id: I2b80cc8fe141f062c2a2bef17733ac69b24e4a48
Reviewed-on: https://gerrit.instructure.com/153473
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-06-12 17:07:03 +00:00
Steve Kacsmark 490674dd41 Cleanup stale imports, Update UI while importing
refs QUIZ-4583, QUIZ-4626

Must be tested in conjunction with QUIZ-4415

Provides visual indicators in assignment list for the Importing, and
Failed to import states. Allows for the user to remove a failed
assignment import from the list.

Test Plan:
  - Do a Common Cartridge or QTI import, enabling Quizzes.Next importing
  - Monitor state transitions in Canvas Assignment views.
  - Ensure they move from Importing to a finished stated.

Change-Id: I6f07ecae7b5a93dadd5309bb54bf15724f512c16
Reviewed-on: https://gerrit.instructure.com/151026
Reviewed-by: Omar Khan <okhan@instructure.com>
Tested-by: Jenkins
QA-Review: David Tan <dtan@instructure.com>
Product-Review: Kevin Dougherty <jdougherty@instructure.com>
2018-06-07 22:21:59 +00:00
Matt Sessions 7286e89be7 Create alert for assignment_missing
Refs MBL-10482

Test Plan:
 - Create a super observer connected to a student
 - Student should be enrolled in a course
 - Create a threshold for `assignment_missing` for that student
   - POST /api/v1/users/observer_alert_thresholds
   - {
       student_id: :student_id,
       threshold: {
        alert_type: 'assignment_missing'
       }
     }
 - Create an assignment that is due in the course for a minute
   or two in the future
 - Wait at least 5 minutes
 - As the observer hit /api/v1/users/self/observer_alerts/:student_id
 - There should be an alert for that user

Change-Id: If2dea94052ff932c866a7382a2119a3287350c28
Reviewed-on: https://gerrit.instructure.com/152073
Reviewed-by: Cameron Sutter <csutter@instructure.com>
Product-Review: Cameron Sutter <csutter@instructure.com>
Tested-by: Jenkins
QA-Review: Taylor Wilson <twilson@instructure.com>
2018-06-07 22:16:26 +00:00
Rob Orton 69ef0be0bb create sis_roll_back_data
fixes CORE-931

test plan
 - migrations should run

Change-Id: I0e836bb4db2ab19fd7763864bea02c8c9dcbe552
Reviewed-on: https://gerrit.instructure.com/139665
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
QA-Review: Rohan Cheeniyil <rcheeniyil@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2018-05-29 21:27:11 +00:00
Cody Cutrer 195d2370e2 fix shard settings deserialization during boot
of unexplained circumstances 🤷

Change-Id: I28dfe553c97ad1df916c0b3cefd143a197b15c48
Reviewed-on: https://gerrit.instructure.com/151694
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-05-29 17:28:54 +00:00
James Williams e293b2ca65 transform id columns when calling pluck on array
Change-Id: I0c8e8417bcead9d9e1243dabae1444fae3e3634b
Reviewed-on: https://gerrit.instructure.com/150278
Reviewed-by: Mysti Sadler <mysti@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2018-05-24 14:42:23 +00:00
Matt Sessions b6f7be8f8c Delete alerts periodically
Refs MBL-10126

Test Plan:
 - In the rails console, create a
   UserObservationLink and an ObserverAlertThreshold
 - Create an alert that links to both
 - Create another alert that links to both but has
   a created_at set to more than 6 months ago
 - Run ObserverAlert.clean_up_old_alerts
 - Run ObserverAlert.all
 - You should only have 1 alert

Change-Id: I23f4723574cde01135ddafb9926c06029266d07d
Reviewed-on: https://gerrit.instructure.com/150324
Reviewed-by: Cameron Sutter <csutter@instructure.com>
Product-Review: Cameron Sutter <csutter@instructure.com>
Tested-by: Jenkins
QA-Review: Kausty Saxena <ksaxena@instructure.com>
2018-05-19 01:42:42 +00:00
Simon Williams ebbfde74ef make `wait_for_replication` more robust
we discovered that direct string comparision of xlog locations can give
invalid results in some situations, where second part of the xlog
location value "wraps" from 7 hex characters to 8

Change-Id: I62215e5fcc3f9655643aab9fc99f55ddbb37e8ac
Reviewed-on: https://gerrit.instructure.com/150385
Tested-by: Jenkins
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2018-05-17 19:02:01 +00:00
Cody Cutrer 8aa20f1066 infer region name, and pre-set instance id for jobs autoscaling
Change-Id: Iac1df05fab63902a12d852d7a55ed728d6e3b113
Reviewed-on: https://gerrit.instructure.com/149709
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-05-11 19:22:42 +00:00
Cody Cutrer 7840c26abf allow enabling inst-jobs health checks
Change-Id: Ic7f8106a9f63c585b8a618b052d82f0a1f1a6312
Reviewed-on: https://gerrit.instructure.com/149579
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-05-11 19:22:32 +00:00
Cody Cutrer f741d3a6fa use switchman-inst-jobs, and drop duplicate code
Change-Id: Ieb1ec1c0f04587959c7b13c6ff3ad30960012b33
Reviewed-on: https://gerrit.instructure.com/149502
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-05-11 15:53:57 +00:00
Cody Cutrer 6308090019 properly quote stuff in find_ids_in_ranges
fixes problems when run on relations that are using `from()`

Change-Id: I528dfb7e6885b4ce276900a53d9b2f8adfabafe5
Reviewed-on: https://gerrit.instructure.com/149795
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-05-10 02:48:53 +00:00
Simon Williams b4fde1f11b used ignored_columns instead of DROPPED_COLUMNS
closes CORE-1245

recent internal changes in rails (released in 5.1.6) make it more
difficult for our module prepend to work, because column information is
no longer referenced lazily when checking on attribute defaults (see
changes to model_schema.rb). this means that we would need to tie in at
a lower level so that _default_attributes excludes the dropped column,
but in looking at that, Rails 5 added support for ignoring columns,
which ties in at the right level and should solve the same use case we
have.

see http://api.rubyonrails.org/classes/ActiveRecord/ModelSchema/ClassMethods.html#method-i-ignored_columns-3D

Change-Id: I1f76e9d0894551f90232a36f5d48f27f8989c3a1
Reviewed-on: https://gerrit.instructure.com/145723
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2018-05-04 20:53:26 +00:00
Omar Khan 391585bd9b Mark old duplicating assignments as failed
We're planning to make API calls to Canvas from Quizzes.Next to notify
it that an assignment has finished duplicating. But making API calls to
Canvas is hard, as it requires lots of configuration, an oauth dance,
and a fair amount of luck.

To ensure that assignments don't get stuck in the "duplicating" state if
Quizzes.Next -> Canvas communication breaks down, this commit adds a
periodic job to transition assignments to the "failed_to_duplicate"
state if they have been duplicating for more than 5 minutes.

Refs QUIZ-3764

Test plan:

- Create a Quizzes.Next assignment
- Go to the canvas assignments page
- Click on the kebab menu for the assignment, click "Duplicate"
- Wait 10 minutes
- Refresh the page
- Check that the new assignment displays a "failed to duplicate" message

Change-Id: Ibaf8c16117bc7ab7fc24007944943c3090ba7041
Reviewed-on: https://gerrit.instructure.com/148001
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Michael Hargiss <mhargiss@instructure.com>
Product-Review: Omar Khan <okhan@instructure.com>
2018-04-26 18:46:49 +00:00
Cody Cutrer 41794a9383 rename AccountAuthorizationConfig* to AuthenticationProvider*
I couldn't take it anymore. it was driving me crazy

Change-Id: Ib40addff56e98a6e0a6d80f246c208e0ce0cb762
Reviewed-on: https://gerrit.instructure.com/148026
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-04-26 16:03:47 +00:00
James Williams 7423e4fb94 rails 5.2
closes #CORE-1301 #CORE-1302

Change-Id: I687132b066d12cf35e3c7a593dff9b2a91337f33
Reviewed-on: https://gerrit.instructure.com/147220
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2018-04-25 19:15:20 +00:00
Simon Williams 6b690371d2 remove `save_without_callbacks`
rails uses callbacks internally to manage dirty state tracking. in
a recent update, skipping these callbacks when saving a new object
breaks any future updates to that object (because id_from_database is
not populated, which is then used as `nil` when trying to do
a subsequent update). see:
6932998fc7 (diff-1a091d0f7eb7b0b2ca95c82f5ba0e1fb)

Because of that I feel like saving without any callbacks is somewhat
dangerous. I think if you have to do it, use one of the built in active
record methods that skips callbacks:
http://guides.rubyonrails.org/active_record_callbacks.html#skipping-callbacks

Or use our active_record-suspension gem to skip specific app-level
callbacks.

test plan: tests pass

Change-Id: I7bfb87e328317430fb2a77edca56cace2cd372b5
Reviewed-on: https://gerrit.instructure.com/146139
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2018-04-10 19:14:44 +00:00
Cody Cutrer cb4fb9468e prevent future migrations
Change-Id: I10febc85e0826679da77c8627d3c49d690a68192
Reviewed-on: https://gerrit.instructure.com/145943
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-04-05 16:02:36 +00:00
Simon Williams 73e3a30bb6 always use current time for migration version
we never want to just increment the last migration version number by 1,
it's too likely to generate conflicts

Change-Id: If6c95470b9fae257cc40c45a3eac0e1d871fb657
Reviewed-on: https://gerrit.instructure.com/145952
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2018-04-05 15:56:25 +00:00
Rob Orton e1b9a42062 create parallel report runners
fixes CORE-1240

test plan
 - run account report using parallel runner
 - it should work

Change-Id: Ib0e8480e53d65b7e5b7f4d3c3f2f36c7d283fb52
Reviewed-on: https://gerrit.instructure.com/145605
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2018-04-04 19:16:44 +00:00
James Williams 22472e9428 don't call current_xlog_location in a transaction
Change-Id: I67b0b7b9baa41d76a34f37e2b33541cb10760dc6
Reviewed-on: https://gerrit.instructure.com/145006
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2018-03-28 19:54:41 +00:00
Cody Cutrer 3320e306a3 disallow deprecated Dirty methods in Rails 5.1
closes CORE-1196

Change-Id: I9a26a8748a61365248b38b6268a6c8bfb3a783d8
Reviewed-on: https://gerrit.instructure.com/144686
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2018-03-23 22:03:20 +00:00
Cody Cutrer 610751f33a vacuum stream_items an stream_item_instances after cleaning them up
also do it 4x daily rather than 1x daily

Change-Id: I328091586226e334c48bfe96b57b0a601bb22ef2
Reviewed-on: https://gerrit.instructure.com/143678
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-03-22 21:56:36 +00:00
Ryan Shaw d8014ed53c make feature flags for Right To Left layout
Closes: CORE-1154

There are 2 feature flags here. One a specific user (internal
employees, sales people, contractors, etc) can turn on to see where we
are and what would still need to be done. And the other is for an
account to be able to turn on to let the users in their account that
have their language set to an RTL language (eg: Arabic, Farsi or Hebrew)
see the interface in RTL

This commit is part of a chain of commits. you'll need to apply the ones
that follow-on after this to see most of the actual Right To Left
features, this commit is mainly just to create the feature flag.

Test plan:
* go to your user settings page
* you should see a feature option for Force RTL Layout”
* it should show that it is “in development”
* it should should be hidden: eg: only appears for site admins and not
  For normal users
* turn it on

(if you have this checked out along with the others that follow beind
  this commit)
* navigate around canvas, most things on most pages should be RTL
* you should be able to get an idea of how much more work and what
  Kind of changes we’d still need to make to get it 100%

* now turn off that user feature flag, and go to your account's settings
* turn on the "allow users from RTL languages to see RTL layout"
  account-level feature flag.
* nothing should change for you, it should be LTR like normal
* set your language to either Arabic, Farsi or Hebrew
* now the interface should all be in right to left.
  (again, you'll need to have the follow-on commits to this to see all
  of the RTL changes)

* without either of the feature flags turned on, absolutely nothing
  should be different at all anywhere in the interface.

Change-Id: Ia78d86fa46d2d289b1913c5d1340546d7f6c3a1f
Reviewed-on: https://gerrit.instructure.com/143891
Reviewed-by: Brent Burgoyne <bburgoyne@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Ryan Shaw <ryan@instructure.com>
Tested-by: Jenkins
2018-03-21 21:09:38 +00:00
Simon Williams c0b7b9ffc3 upgrade to rubyzip 1.2.1
Even though this is a patch level version bump, it made a subtle change
that broke some workflows we had (see https://xkcd.com/1172/).

The second argument to Zip::File.open is a `create` flag, which
determines whether the zip file is created if it does not exist. Prior
to 1.2.1, the expectation was that `Zip::File::CREATE` was passed as the
argument if creation was desired. However, internally the library was
inconsistent about checking the passed flag against `Zip::File::CREATE`
vs just checking truthiness.

In practice, what this meant is that if you passed a different truthy
value, like `Zip::File.open(filename, 'w')` (like we did in several
places), the file would still be created if it did not exist, BUT,
critically, it would not be saved if nothing was put into the archive.

1.2.1 changed this flag to always check "truthiness", meaning empty zip
archives started to be saved. Unfortuantely, our mime type evalutation
rules don't correctly identify an empty zip file as a zip file, but
instead as just a binary file. These two things combined to create
problems.

see https://github.com/rubyzip/rubyzip/compare/v1.2.0...v1.2.1
and particularly fc23f68f77

closes CNVS-41711

test plan:
- test course copy and export/import workflows (both canvas and standard
common cartridge formats), especially in "sparse" cases where, for
example, no associated files would be copied.

Change-Id: If0033f6c0ede7f7a6e8c26031371ae2bd24cd1fb
Reviewed-on: https://gerrit.instructure.com/143744
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Deepeeca Soundarrajan <dsoundarrajan@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2018-03-21 17:09:44 +00:00
Cody Cutrer 616b2dbb47 drop rails 5.0
Change-Id: Ic374d543a7ba00f660efccb287513c5f80232196
Reviewed-on: https://gerrit.instructure.com/144178
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-03-20 19:50:23 +00:00
Cody Cutrer c988523bbe postgres 9.3 is already required
(Canvas won't even boot with an older version)

so remove conditionals for older features

Change-Id: Ied601ef97f527ac93d386a650a1bd23f198dcf04
Reviewed-on: https://gerrit.instructure.com/143634
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2018-03-14 22:25:58 +00:00
Cody Cutrer 5e3038b721 codify our sorting cases in a spec
refs gh-1229

also make clear how the collator is configured, vs. the defaults
also also document each of our three uses of ICU to point to each
other if you change anything in the future

Change-Id: Ib911302a96a9d9a667923793ebb2be095b62d8ab
Reviewed-on: https://gerrit.instructure.com/142772
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2018-03-14 21:05:57 +00:00
Cody Cutrer 4ab57d6217 use correct syntax for renaming a constraint in postgres
Change-Id: Iee92ce11c4840c7443fc619356141930dcc8f36e
Reviewed-on: https://gerrit.instructure.com/143609
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-03-14 20:15:31 +00:00
Rob Orton 52ced9231d don’t delete submissions on user merge
fixes CORE-1069
refs CORE-1026

test plan
 - have 2 users with conflicting submissions
   one with grade one with submission
 - merge them
 - split them and it should be restored

Change-Id: Id645ddcc0c5182a825e513de7e7d05e6fa50f254
Reviewed-on: https://gerrit.instructure.com/142004
Reviewed-by: Keith T. Garner <kgarner@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2018-03-14 16:49:52 +00:00
Simon Williams 38c673b7f0 don't send Turnitin::Errors::SubmissionNotScoredError to sentry
this is just used to fail the delayed job, so it retries. if it fails
it's max retries, the error details are logged on the submission

Change-Id: If1f7da5dd5331ad153d78c323526eac22afd01c5
Reviewed-on: https://gerrit.instructure.com/143174
Tested-by: Jenkins
Reviewed-by: Nathan Mills <nathanm@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2018-03-12 20:48:57 +00:00
Rob Orton e68b92b2c2 Revert "fix footer logo in footer of emails"
This reverts commit 2b9853c82d

but replaces the googlusercontent url that doesn't exist on the 
internet with one that will continue to exist:

aka: https://du11hjcvx0uqb.cloudfront.net/dist/images/email_signature-ea0b2cda73.png

Change-Id: I82ba0bb3cc5ae3ef4749f1fcaabe783775dcfc78
Reviewed-on: https://gerrit.instructure.com/143074
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Reviewed-by: Ryan Shaw <ryan@instructure.com>
Product-Review: Ryan Shaw <ryan@instructure.com>
QA-Review: Ryan Shaw <ryan@instructure.com>
Tested-by: Jenkins
2018-03-09 00:16:53 +00:00
Jacob Fugal 83fed6665c define account method on delayed jobs correctly
test-plan:
- without any other plugin defining `#account` on delayed jobs,
  `job.account` should still exist and should return Account.default
- with some other plugin already defining `#account` on delayed jobs,
  `job.account` should still use the plugin defined behavior

Change-Id: I5c74febd4525b4196e38c632cfb737a97090f704
Reviewed-on: https://gerrit.instructure.com/143005
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Michael Jasper <mjasper@instructure.com>
QA-Review: Michael Jasper <mjasper@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
2018-03-08 22:07:01 +00:00
Steven Burnett 2b9853c82d fix footer logo in footer of emails
fixes COMMS-939

Test Plan:
- send any email in canvas
- notice the footer image is there
- inspect the html source, and notice that the host is
  your local Canvas install
- enable CDN config, run `node_modules/.bin/gulp rev`,
  and `rake canvas:cdn:upload_to_s3`
- send another enable
- notice the footer image is still there
- inspect the html source, and notice that the host
  is the CDN
Change-Id: Ieaed838dee02168a2aa67f3448d8d68fb6f5b701
Reviewed-on: https://gerrit.instructure.com/142889
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Landon Gilbert-Bland <lbland@instructure.com>
QA-Review: Gentry Beckmann <gbeckmann@instructure.com>
Product-Review: Gentry Beckmann <gbeckmann@instructure.com>
2018-03-08 17:53:17 +00:00
Jayce Higgins 4121436abc Create quizzes next migration service for cc export
closes: QUIZ-3861, QUIZ-4113

Test-Plan:
  - QA-CR because it's not hooked up yet

Change-Id: I9028c91f485b63bc8ca8d2c70567c78945c2a848
Reviewed-on: https://gerrit.instructure.com/142304
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Jayce Higgins <jhiggins@instructure.com>
2018-03-07 18:37:59 +00:00
Jacob Fugal 171ef86f81 set Attachment.current_root_account during jobs
and rename that suite of methods to make more sense

test-plan: N/A

Change-Id: Iffc520ea55141ac47da669663838a4d3c3d8712c
Reviewed-on: https://gerrit.instructure.com/142486
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
2018-03-07 17:18:24 +00:00
Cody Cutrer 1acbe1bc91 make CanvasKaltura.cache aware that Rails.cache is dynamic
Change-Id: I1b1aa95632b4e270d729c69f184e4507899cd3af
Reviewed-on: https://gerrit.instructure.com/142666
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-03-06 05:46:59 +00:00
James Williams 2142fa4fff master courses: copy assignment group rules selectively
should still copy rules correctly even if the referenced
 assignments are not included in the copy

test plan:
* create a blueprint course
* have an assignment group with assignments and
 a dropping rule that excludes one of the
 assignments
* sync to an associated course
* change the rule on the group
* re-sync
* it should still exclude the correct assignments
 in the associated course group

closes #ADMIN-801

Change-Id: I60e2a691426364ca0b93a1f67b29d1ab05d18e8a
Reviewed-on: https://gerrit.instructure.com/141655
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Tested-by: Jenkins
QA-Review: Deepeeca Soundarrajan <dsoundarrajan@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
2018-03-05 16:23:41 +00:00
Cody Cutrer 91cfe36efd skip fancy find_in_batches with eager_load values
fixes GRADE-907, CORE-1050

they don't work right. Rails accomplishes them by selecting IDs
first, and then doing a second query for the eager load. too much work
to re-implement

Change-Id: I0ef2cb437850521c79c22363d82e7302331dc5f7
Reviewed-on: https://gerrit.instructure.com/141865
Reviewed-by: Shahbaz Javeed <sjaveed@instructure.com>
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-02-26 17:57:24 +00:00
Cody Cutrer 7c034266f1 allow avoiding find_in_batches_with_copy
with the :start option

Change-Id: Ibbc577777757a6f2fd2427724e5ea5508a20651f
Reviewed-on: https://gerrit.instructure.com/141862
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-02-26 17:56:57 +00:00
Cody Cutrer 3e82d9136a annotate COPY TO STDOUT queries with marginalia if configured
closes CORE-1027

test plan:
 * configure marginalia.yml
 * in consule, run Account.active.find_each {}
 * observer the COPY query, with a marginalia comment

Change-Id: I7708e641e699e52637a800e1d3f25d9b26d94a7a
Reviewed-on: https://gerrit.instructure.com/141636
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2018-02-22 19:31:34 +00:00
wdransfield 6b465dd804 Fix last N+1 for adding originality_data to submission_history
Closes: GRADE-861

Test Plan
- Verify there are no N+1 queries when loading speedgrader
  caused by originality reports
- Verify there are no N+1 queries when loading gradebook
  caused by originality reports
- Verify originality reports are copied to all submissions
  in a group when they are created or updated in a group
  assignment.
- Verify online upload originality reports are displayed
  correctly in speedgrader, gradebook, student grade page,
  and submission details page.
- Verify originality reports display properly in gradebook
  and speed grader when there are multiple submissions
  and each has a different originality report.
- Verify the "resubmit to plagiarism tool" button still
  appears in speedgrader.
- Verify originality reports for group assignments are displayed
  correctly.
- Verify text entry originality reports are displayed correctly.
- Verify originality report launches work correctly from speedgrader
  and gradebook.

Change-Id: I459bbf0a165bb131d58fc153feb0dce3d0bf07af
Reviewed-on: https://gerrit.instructure.com/141321
Tested-by: Jenkins
Reviewed-by: Keith T. Garner <kgarner@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Weston Dransfield <wdransfield@instructure.com>
2018-02-22 17:01:41 +00:00
Jeremy Stanley 118c552002 allow unicode characters submission zip files
test plan:
 - have an assignment that accepts file uploads
 - have a student with non-ASCII characters in the name
   (e.g., accented or Japanese, etc.)
 - submit as the student
 - as the teacher, on the assignment page right sidebar,
   download submissions (ensure jobs are running)
 - the non-ASCII characters should not be filtered out
   of the student's name in the filename

notes:
 - spaces and commas and hyphens, etc., will still
   be filtered out of names; this was done for
   other reasons and has not changed. the difference
   is we use [[:word:]] instead of \w, which includes
   word-like Unicode general categories and not just
   alphanumerics
 - windows explorer prior to windows 8 does not
   properly support unicode filenames in zip files.
   there's not much we can do about this, other than to
   suggest using a third-party zip program that does.
   (ASCII filenames are still handled correctly)
 - the student's name is not included if anonymous grading
   is turned on

fixes ADMIN-770

Change-Id: I1cb06a4277ee330e02dabab1d7d7cae01d80822a
Reviewed-on: https://gerrit.instructure.com/140996
Reviewed-by: Ed Schiebel <eschiebel@instructure.com>
Reviewed-by: Jon Willesen <jonw+gerrit@instructure.com>
Tested-by: Jenkins
QA-Review: Jon Willesen <jonw+gerrit@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
2018-02-21 16:08:36 +00:00
Cody Cutrer 761723cdad move TypeMapInitializerExtension earlier in the boot process
so that the connection to the default shard catches it. also
stop using the Rails 4 location of it

Change-Id: Iec213127458d77d12b98fd9a8f02fec7f74f1cc4
Reviewed-on: https://gerrit.instructure.com/141434
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-02-21 00:03:55 +00:00
Cody Cutrer c68bfe4e5a detect transactions in dev for find_in_batches_with_temp_table
Change-Id: I1f7a3185c8146ae45aeec416c9403881d9f3f210
Reviewed-on: https://gerrit.instructure.com/141101
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-02-15 20:28:34 +00:00
Cody Cutrer 80613cfcba add explain(analyze: true) support
also, allow explain to take a block, so you can run it for any query
(i.e. User.explain { something_complicated.count })

Change-Id: Ie724215ed1f1a45e7bec49a6deb960d09876bf03
Reviewed-on: https://gerrit.instructure.com/140615
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-02-12 19:46:46 +00:00
Cody Cutrer 09fd728969 allow aliases for locale names coming from browsers
fixes gh-1167

and set them up for the chinese variants

test plan:
 * change your browser language to Chinese (Traditional)
 * ensure your account and user don't override the language
   (both set to System Default)
 * inspect your web request, make sure the accept-language includes
   zh-TW
 * you should see Canvas in traditional chinese, not simplified
   chinese

Change-Id: Id09e9b348c18195809b96138d081c246c080fa29
Reviewed-on: https://gerrit.instructure.com/133838
Reviewed-by: Brent Burgoyne <bburgoyne@instructure.com>
Tested-by: Jenkins
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2018-02-05 19:52:03 +00:00
Cody Cutrer c11f0ed738 implement and prefer to use COPY TO STDOUT for find_in_batches
where possible

fixes CORE-939

test plan:
 * run some account reports
 * check your log
 * see that reports were run with COPY (...) TO STDOUT, and not
   a cursor

Change-Id: Icf953ffe33c12b2046187ceecad0397ce20a9eb4
Reviewed-on: https://gerrit.instructure.com/139552
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2018-01-31 21:11:31 +00:00
Rob Orton 5e735cebeb store sis_batch_errors in the db
fixes CORE-324

test plan
 - run a sis import with errors
 - the errors should be included in the api JSON

Change-Id: I97e372a82b565e7bfa01cd9ca0bb66bb67c95723
Reviewed-on: https://gerrit.instructure.com/139071
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
2018-01-26 22:08:25 +00:00
Jonathan Featherstone 88f74435c7 Fix dynamic_settings reload issue
Test Plan:
  * Configure canvas to work with 3rd party service (like address-book)
  * Confirm that canvas can connect to third party service
  * Save or touch any file (IE app/observers/live_events_observer.rb)
  * Reload page, and confirm that canvas can still connect to service

closes RECNVS-246

Change-Id: Ib3a1b12cdc038f90c139e4d4c957e960288b3e9a
Reviewed-on: https://gerrit.instructure.com/138638
Tested-by: Jenkins
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Collin Parrish <cparrish@instructure.com>
Product-Review: Jonathan Featherstone <jfeatherstone@instructure.com>
2018-01-24 21:28:36 +00:00
Rob Orton 936d0fbf48 update remove_foreign_key_if_exists?
refs CORE-899

test plan
 - migrations should run

Change-Id: I7ab923cc917f753b92fbaf69b3e910f599ecc052
Reviewed-on: https://gerrit.instructure.com/138834
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2018-01-24 04:28:45 +00:00
Derek Bender c966d809aa Revert "enable better_errors for development part two"
This reverts commit ba72375012.

Change-Id: I6b086a276c00abbebbe781ec42c1a43e3f3cf8fb
Closes: CNVS-40917
Reviewed-on: https://gerrit.instructure.com/137873
Tested-by: Jenkins
Reviewed-by: Spencer Olson <solson@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
QA-Review: Eric Coan <ecoan@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
2018-01-13 00:50:47 +00:00
Cody Cutrer ac6003004f remove unused method
Change-Id: Icca8d162292af8d6a428ab84a5a8c0b5b5223df4
Reviewed-on: https://gerrit.instructure.com/136754
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2018-01-03 19:54:16 +00:00
Simon Williams c4d5fea9be fix wal location fn names in pg 10
Change-Id: Ieb0faa8b5f0295c035d88499cd6730a432ab5521
Reviewed-on: https://gerrit.instructure.com/136260
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2017-12-20 21:02:10 +00:00
Cody Cutrer b6007d79f3 fix Connection#readonly? on Rails 5+
refs CORE-786

Change-Id: Ic4a0d3b1479168f0d777ac35f0c83fc57c687945
Reviewed-on: https://gerrit.instructure.com/136094
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-12-19 17:10:35 +00:00
James Williams 8ad3f46b3d don't check for insecure uri if ip filter setting is blank
Change-Id: If4e49744d41e9baa54f1575e8f60e236a00e6b4c
Reviewed-on: https://gerrit.instructure.com/134747
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
2017-12-06 20:10:36 +00:00
Cody Cutrer c5a35fa38a scrub old conversation batches
refs CORE-123

Change-Id: Id3181523ffbd574a1cbf4a92aad4999b2d09b3e8
Reviewed-on: https://gerrit.instructure.com/134279
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-12-01 02:14:53 +00:00
Cody Cutrer 189febba41 add additional brazil time zones
fixes CORE-638

also change time zone picker to show DST offset as well as the
base offset if they respect DST in that zone

test plan:
 * go to change your user settings, and in the time zone drop down
   you should see all 6 brazil zones.
 * ensure saving thas these zones actually persists

Change-Id: I2a31780f25f396cc79a8d5a630791646cdda8e76
Reviewed-on: https://gerrit.instructure.com/133060
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-11-28 21:31:57 +00:00
Cody Cutrer 8b9aeeb6b0 fix best_unicode_collation_key varying by db server
also, activate the correct shard before using it in SisPseudonym

Change-Id: If7cc30580630aea190f4ca5853ad25ec3fec4e2d
Reviewed-on: https://gerrit.instructure.com/133876
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-11-28 20:32:25 +00:00
Felix Milea-Ciobanu bcab182ae8 fix inconsistent calendar day order across locales
fixes CORE-47

test plan:
- have a user who has their language set to spanish
- checkout the following locations and ensure that week days run from
  Monday to Sunday:
  - syllabus mini calendar
  - calendar page week + month view
  - calendar page mini calendar
  - date picker on assignment or calendar event
- have a user ho has their language set to english
- check the same locations and ensure that week days run from Sunday
  to Saturday

Change-Id: I03d02f37aa6c5570967b092ffcdd774acc62984b
Reviewed-on: https://gerrit.instructure.com/132264
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Collin Parrish <cparrish@instructure.com>
Product-Review: Felix Milea-Ciobanu <fmileaciobanu@instructure.com>
2017-11-09 23:16:49 +00:00
Derek Bender ba72375012 enable better_errors for development part two
This re-enables better_errors with an opt in policy for
binding_of_caller (which powers the browser REPL) to safeguard
by defautl against RCE with tools like ssh, ngrok or localtunnel

The key for enabling the console is BETTER_ERRORS_ENABLE_CONSOLE
and the key for disabling better_errors altogether is
BETTER_ERRORS_DISABLE.

Docker users: set your TRUSTED_IP address in your environment
variables to access better errors in you web container.

fixes: GRADE-156

test plan:
 - by default, no live REPL console should be present when the
   error page is present (e.g. /accounts/0)
 - given BETTER_ERRORS_ENABLE_CONSOLE=true
 - the live REPL console is now present
 - given BETTER_ERRORS_DISABLE=true
 - the better errors page is not displayed (e.g. /accounts/0)

Change-Id: Ia259bc466136d678aa0b33047dda5f2aa0b3c28d
Reviewed-on: https://gerrit.instructure.com/95120
QA-Review: Keith T. Garner <kgarner@instructure.com>
Tested-by: Jenkins
Reviewed-by: Keith T. Garner <kgarner@instructure.com>
Product-Review: Keith T. Garner <kgarner@instructure.com>
2017-10-23 16:09:33 +00:00
Cody Cutrer 9d3b2190ba add Philippines as a friendly time zone
closes CNVS-39780

test plan:
 * change your time zone to Philippines

Change-Id: Ief15405da51f0471c2bd10a239ef305ae2b80a18
Reviewed-on: https://gerrit.instructure.com/129722
Tested-by: Jenkins
Reviewed-by: Clay Diffrient <cdiffrient@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2017-10-18 16:58:43 +00:00
James Williams 14d2d4d495 drop appointment_group context columns
Change-Id: If820d1c671036f5335ec23bc1007626f0b54e265
Reviewed-on: https://gerrit.instructure.com/129712
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-10-16 13:19:34 +00:00
James Williams 8e748faa43 master courses: remove old import_results column
all of them should be migrated over now

Change-Id: I4663d44ccafd913087ecef3cf89809473e5be202
Reviewed-on: https://gerrit.instructure.com/129707
Tested-by: Jenkins
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-10-16 12:16:56 +00:00
Cody Cutrer e63830d251 drop jobs max attempts default to 1
closes CNVS-38713

Change-Id: I115d7d51493b2781f04f8c0bf71e157d808825b3
Reviewed-on: https://gerrit.instructure.com/129709
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-10-14 17:57:03 +00:00
Cody Cutrer 68e79d909b fix migration version number generation
ignore the brand config versions that aren't dates

Change-Id: Iecc774799e3ed7769a9590766e46f77d23ef2f41
Reviewed-on: https://gerrit.instructure.com/129055
Tested-by: Jenkins
Reviewed-by: Ryan Shaw <ryan@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-10-10 17:05:22 +00:00
Ryan Shaw a9eb19ffa6 make compile_assets work when there’s no db.yml
Fixes: CNVS-39854

Test plan:
* mv config/database.yml config/database.yml.backup
* bundle exec rake canvas:compile_assets
* it should work
* Docker build should pass

Change-Id: I0786d1de1d6f6f69d665e10b0428571e77d7f859
Reviewed-on: https://gerrit.instructure.com/128974
Reviewed-by: James Williams  <jamesw@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Ryan Shaw <ryan@instructure.com>
QA-Review: Ryan Shaw <ryan@instructure.com>
2017-10-09 16:13:31 +00:00
Jeremy Slade fbd50a83c6 Pass correct args to chained :perform callbacks
Closes: CNVS-39627, AMS-701

The Delayed::Worker around callbacks are chained,
so all of the required parameters must be passed
through to the block in case it is another callback
that relies on them.

Change-Id: I740d42bfc36e3dda52c85802ddca673f2b2f6bb8
Reviewed-on: https://gerrit.instructure.com/128232
Tested-by: Jenkins
Reviewed-by: Dave Donahue <ddonahue@instructure.com>
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Product-Review: Jeremy Slade <jslade@instructure.com>
QA-Review: Jeremy Slade <jslade@instructure.com>
2017-10-04 15:27:57 +00:00
Cody Cutrer b87b523776 reload dynamic_settings.yml on HUP
Change-Id: I7b15834655b66c1114e301336b7e673664000fdb
Reviewed-on: https://gerrit.instructure.com/127231
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-10-02 14:56:20 +00:00
Cody Cutrer ad73cba8bd if no config changed, don't forget the old config
Change-Id: I24509dce1a218e67d8438c5405506dcdc711cd33
Reviewed-on: https://gerrit.instructure.com/126702
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-09-20 15:24:05 +00:00
Cody Cutrer 92ce7b8b88 protect against SIGHUP hooks crashing existing processes
also change cache_store reloader to not leave a bad state
if one cluster has invalid config

Change-Id: I711aff6d9dbbadff8a6ea66a6f438781b5c8e19b
Reviewed-on: https://gerrit.instructure.com/126634
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-09-19 22:19:29 +00:00
James Williams 6403dbe4a0 rails 5.1: fix masquerading rendering
Change-Id: I1703075e45c377a86bdee3757b967e732d1fa318
Reviewed-on: https://gerrit.instructure.com/125926
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-09-14 19:14:39 +00:00
Cody Cutrer 55e13c2b16 fix typo in comment
Change-Id: I0adde048ff57c4806892c2291b1f988209f58814
Reviewed-on: https://gerrit.instructure.com/125931
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
QA-Review: Rob Orton <rob@instructure.com>
2017-09-13 20:39:34 +00:00
Cody Cutrer 6f21e2a961 don't run reloader from trap context
fixes CNVS-39296

test plan:
 * run jobs/web (must be using a web server that doesn't capture HUP
   itself)
 * HUP your processes
 * notice that they don't crash, and the logs will say that the reloader
   ran

Change-Id: I150729fdb68c0fd0ff462d794eecfc8da7e17e5e
Reviewed-on: https://gerrit.instructure.com/125968
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-09-13 20:08:42 +00:00
Cody Cutrer a460fcc37d configure cache store from consul
fixes CNVS-39235

also allow chaining of clusters (to share a cache store among
multiple clusters), and hot-reloading. but do it all carefully so that
if config doesn't change, we don't reconnect, and shared cache stores
are actually shared (not multiple connections)

Change-Id: I2b72472b4d0f4a34619878752801dd5c05a305a8
Reviewed-on: https://gerrit.instructure.com/125833
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-09-13 17:30:48 +00:00
Cody Cutrer a01fad2f19 Revert "partition cache usage by "legacy" and "per-cluster""
This reverts commit bec1612696.

Change-Id: I072a69f0742b29a39930b16e782c1a314b0a3e8e
Reviewed-on: https://gerrit.instructure.com/125837
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-09-13 03:54:32 +00:00
Cody Cutrer 7197e44389 refactor and impove DynamicSettings
closes CNVS-35834

 * allow specifying tree, service, and cluster for consul stuff
 * check multiple consul keys for each setting (cluster, env, region, global)

test plan:
 * an existing consul environment still works

Change-Id: I48e8fadeac2e140973bfc4b41c1cfb386532d15c
Reviewed-on: https://gerrit.instructure.com/125271
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
2017-09-12 22:34:21 +00:00
James Williams 3855fc4af1 filter out localhost and local ips from canvashttp
test plan:
* use the API to queue a content migration for a course
 with a parameter 'file_url' set to a local url
 (i.e. 'localhost:3000/500.html')
* it should not download the file and allow the file
 to be downloaded on the content migrations page
 for the course

* create a link to a local file
 in a piece of course content (i.e. a page)
* run the course link validator
* it should not actually check if the link exists locally
 or not, but rather always flag the link

closes #SEC-606 SEC-607

Change-Id: I671c017ec93d88446df77d716725fff8874622bc
Reviewed-on: https://gerrit.instructure.com/125118
Tested-by: Jenkins
Reviewed-by: Brad Horrocks <bhorrocks@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Heath Hales <hhales@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
2017-09-12 20:29:56 +00:00
Simon Williams d61ccaad3f restore mail handler jobs as singletons
Change-Id: I89a76098f8785a800561c6ec6c5dfbd54f729995
Reviewed-on: https://gerrit.instructure.com/125230
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2017-09-07 19:24:11 +00:00
Cody Cutrer bec1612696 partition cache usage by "legacy" and "per-cluster"
and use per-cluster for permissions

Change-Id: Ib5a84f2768c5e86f2071dbd149e8d9cdfd015917
Reviewed-on: https://gerrit.instructure.com/124951
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-09-06 16:21:13 +00:00
Steven Burnett 82651dc776 add a setting to the footer of emails
fixes CNVS-38989

Test Plan:
- Send yourself an email (one which you can reply too
  aka conversation messages/discussions)
- notice you see the text:
  "You can reply to this message in Canvas by replying directly
   to this email, or by clicking this link: %{link}."
- stop your server and modify config/outgoing_mail.yml
  under development(or whatever env you are in) add
  `reply_to_disabled: true`
  after `  default_name: "Instructure Canvas"` key
- resend yourself an email like above
- notice that text above no longer states you can reply to the email.

Change-Id: I2707b77d3aa296dce5f04359c8e016d805e7ecd3
Reviewed-on: https://gerrit.instructure.com/124813
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Heath Hales <hhales@instructure.com>
Product-Review: Christi Wruck
2017-09-05 18:55:38 +00:00
Jonathan Featherstone 68d1fbc138 Run notification failure and bounce in all regions
Test Plan:
  * Specs

closes CNVS-38891

Change-Id: Ide88ebba414b2c42d8fb35c90f47b8ff157bd2e4
Reviewed-on: https://gerrit.instructure.com/124210
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Collin Parrish <cparrish@instructure.com>
Product-Review: Jonathan Featherstone <jfeatherstone@instructure.com>
2017-09-01 19:47:06 +00:00
Cody Cutrer 6c02a98d05 pull in and allow configuration of inst-jobs-autoscaling
closes CNVS-38845

Change-Id: I580c8f16fb1aab9e48cbb2b6af975fb2ad570a48
Reviewed-on: https://gerrit.instructure.com/124001
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-08-28 22:35:05 +00:00
Simon Williams ce5e54076b allow disabling crocodoc polling job
closes CNVS-38714

test plan:
- enable the crocodoc plugin with a bogus api key
- notice that the CrocodocDocument.update_process_states runs and fails
- check the box to disable polling in the plugin
- the job should stop failing

Change-Id: I365ceeba2d83e4d2ec66ed8e977ba7798a58a960
Reviewed-on: https://gerrit.instructure.com/123017
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Matthew Wheeler <mwheeler@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2017-08-22 18:26:13 +00:00
Simon Williams affa953f61 drop Message#asset_context and DelayedNotification#asset_context
name did not match usage, and all remaining usage can be satisfied in
other ways.

closes CNVS-38407

test plan: regression test message sending (immediate, delayed, and
dashboard notifications w/ stream items)

Change-Id: Ibe110527b4644cbaa417c53b2b53c2fd7e18dc9e
Reviewed-on: https://gerrit.instructure.com/120520
Tested-by: Jenkins
QA-Review: Heath Hales <hhales@instructure.com>
Reviewed-by: Matt Smith <msmith@instructure.com>
Reviewed-by: Benjamin Christian Nelson <bcnelson@instructure.com>
Reviewed-by: Jon Willesen <jonw+gerrit@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
2017-08-22 17:47:51 +00:00
August Thornton f2bc3687e9 convert params to "unsafe" hash before filtering
fixes PLAT-2770

test plan:
• Go to any account/course settings page
• Click on the Apps tab
• Click on any LTI that populates within the App Center (I use 3DGameLab)
• Click '+ Add App'
• Click the 'Add App' button in the window that appears
• Verify app is added successfully

Change-Id: I060ed314423c841ff5d8ef12c46f2377f31b58b1
Reviewed-on: https://gerrit.instructure.com/122918
Tested-by: Jenkins
Reviewed-by: Andrew Butterfield <abutterfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
Product-Review: August Thornton <august@instructure.com>
2017-08-16 16:50:40 +00:00
Omar Khan 36c714c7a6 Add Module#class_name
This method is already defined by YARD, but we don't include YARD in our
production dependencies, so we have to reimplement it.

Test plan: check that production builds work

Change-Id: I1dc383b8ca5414a57151ac409795e53932f9cd8a
Reviewed-on: https://gerrit.instructure.com/122078
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Jon Jensen <jon@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
Tested-by: Jenkins
2017-08-09 14:58:45 +00:00
Cody Cutrer 353bf2ec16 add first class support for the UK Federation
fixes CNVS-38446

refactor InCommon to be a common base class for both, and in our
special casing of InCommon refactor to iterate the children
classes

test plan:
 * add a SAML config with either InCommon or UK Federation special
   metadata URIs, and a valid entity ID for a school that's part of
   one or the other
 * the rest of the metadata should populate without error

Change-Id: Ie2d37e942cc35333ee16d1f0b3cbb2dc81f53767
Reviewed-on: https://gerrit.instructure.com/121421
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
2017-08-08 20:26:52 +00:00
Cody Cutrer c3e4d6630e don't allow vanilla polymorphic: true
Change-Id: I7bde198887b111ac0883fe923eb45bd690426e7e
Reviewed-on: https://gerrit.instructure.com/121040
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-08-02 15:13:53 +00:00
Cody Cutrer ff5b0d3a9d fix return value of delete_all with joins
Change-Id: I648cf30f067d181d7cca757ff27d481966dfeea3
Reviewed-on: https://gerrit.instructure.com/121155
Reviewed-by: James Williams  <jamesw@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-08-01 21:06:40 +00:00
James Williams 0638da43ba rails 5.1: all the rest of the specs
Change-Id: I2f4ca4a671a0e5bdcfbeb90b4714465ec34acc9f
Reviewed-on: https://gerrit.instructure.com/120891
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-07-31 19:48:51 +00:00
Neil Gupta 507b2c3d3d Fix periodic jobs to not use deprecated delete_all syntax
Test plan:
Periodic jobs run

Change-Id: Iecd5fc2660f7daa7da355cb3cd04cbadc8883178
Reviewed-on: https://gerrit.instructure.com/120806
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Neil Gupta <ngupta@instructure.com>
QA-Review: Neil Gupta <ngupta@instructure.com>
2017-07-28 16:22:06 +00:00
Cody Cutrer 332adf2d9e spec: mocha => rspec-mocks for models
Change-Id: I2521e974728e1a0bd6e28717f691b96d81a1f477
Reviewed-on: https://gerrit.instructure.com/120676
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-07-27 16:38:19 +00:00
Simon Williams c5ba9dd6bd drop messages.asset_context_code, unused
closes CNVS-38367

test plan: send a message, it should work

Change-Id: Ic1b99120ee0c93155a4a4475f4d1dfb3d373be37
Reviewed-on: https://gerrit.instructure.com/120424
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Matt Smith <msmith@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2017-07-27 15:05:53 +00:00
James Williams a354602ea8 rails 5.1: bunch of other specs
Change-Id: Ic6e4f64874021639f5e8950e2fe42f714ae31250
Reviewed-on: https://gerrit.instructure.com/120225
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-07-27 12:35:30 +00:00
James Williams 2fae336d7d rails 5.1: prevent uniq calls on relations/collections
Change-Id: I367c6dfbe237272f187e262e86725038fc9febbd
Reviewed-on: https://gerrit.instructure.com/120553
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-07-26 16:33:32 +00:00
Tyler Pickett 1c06002d4c Bump Oj gem and adjust how we're using it
Fixes: CNVS-37742

Test Plan:
* API stuff should still work

Change-Id: If45c36967ffa382b15ff865262e927cd7384d89d
Reviewed-on: https://gerrit.instructure.com/119546
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
2017-07-24 19:18:33 +00:00
James Williams 4ba94cb171 rails 5.1: lib specs
Change-Id: Ieef3d3447cfa2b22c9c9f6662688a40ae48a726c
Reviewed-on: https://gerrit.instructure.com/119730
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-07-20 17:26:51 +00:00
Tyler Pickett 465afda22f Add a flag to completely disable caching of permissions
Fixes: CNVS-38111

Test Plan:
* Disable permission caching
  `Setting.set('permissions_cache_enabled', 'false')`
* Start up your web server and visit any page
* You should not see any Redis requests for permissions in your
  development logs

Change-Id: Ib91a0b7fc9ca8db121f942984e26f44d19d2400b
Reviewed-on: https://gerrit.instructure.com/118850
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
Tested-by: Jenkins
2017-07-18 12:41:34 +00:00
Tyler Pickett 41a66e4d49 Allow us to disable caching of intermediate permissions
Fixes: CNVS-38113

By intermediate permissions we mean permissions used to determine if the
directly queried one matches, for example in `app/models/account.rb` we
have a :create_tool_manually permission that checks if the user can
:lti_add_edit. Before this patch both permissions would get written to
the Rails cache, with this patch and the flag set to false :lti_add_edit
won't get written but the top level :create_tool_manually will.

Test Plan:
* Set the new flag to false:
  `Setting.set('permissions_cache_intermediate', 'false')`
* Start tailing the development log grepping for setting permissions
  `tail -f log/development.log | grep 'setex permissions'`
* Create an LTI1 tool registration using the UI, you should see a
  permission named `create_tool_manually` get cached while one named
  `lti_add_edit` should not.

Change-Id: I88ba5cc03d1a13030f53194554d6eee3c287c10d
Reviewed-on: https://gerrit.instructure.com/118656
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
2017-07-17 17:11:54 +00:00
Shahbaz Javeed c2cacc5b23 recreate the pg_collkey based indexes to use collation level 3
closes CNVS-38143

this commit aligns the collation schemes for postgres, ruby and
javascript to the same level.  this means that when sorting an
array of strings in any of the three environments, the result
will be identical.

this also stops relying on default collation strengths in ruby
and postgres and explicitly specifies the default collation
strengths to avoid surprises later.

test plan:
* Verify migrations pass with multiple shards
* Verify migrations rollback properly individually

Change-Id: I6023b364c260c35b85a8d6364ccc346e2ef7d159
Reviewed-on: https://gerrit.instructure.com/118675
Tested-by: Jenkins
Reviewed-by: Derek Bender <djbender@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Keith T. Garner <kgarner@instructure.com>
QA-Review: Keith T. Garner <kgarner@instructure.com>
2017-07-14 21:30:46 +00:00
Cody Cutrer 9786b67804 remove extra logging to debug Passenger connection orphan issue
refs CNVS-37760

Change-Id: I99889adc330c214748719bf60ef3a2e4ff877c0e
Reviewed-on: https://gerrit.instructure.com/118597
Tested-by: Jenkins
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-07-14 16:27:03 +00:00
Tyler Pickett 1393e9e7fc Add setting to disable related permission caching
Fixes: CNVS-38114

We were just blindly caching everything that could be cached when
calculating permissions, it turns out these aren't actually used very
frequently so we're going to try turning caching of them off.

Test Plan:
* With Redis backed caching enabled
* Tail your development log grepping for setting permission caches:
  `tail -f log/development.log | grep 'setex permissions'`
* Visit your main account settings page: `/accounts/1/settings`
* Note that there are many manage, update, and delete permissions
  written to redis.
* In a console disable related permissions caching:
  `Setting.set('permissions_cache_related', 'false')`
* Reload configuration values for your web process (SIGHUP or restart)
* Reload the account settings page, note that there are only redis
  writes for read permissions now instead of many others.

Change-Id: Ic5a73a5124b42405fed79bede37aff70322d0c54
Reviewed-on: https://gerrit.instructure.com/118480
Tested-by: Jenkins
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
2017-07-12 19:22:24 +00:00
Tyler Pickett f1e4031514 Add blacklist for permissions caching
Fixes: CNVS-38110

This is the first of a few knobs we'll be adding to give the ability to
tune rails cache traffic coming from adheres_to_policy. We're adding
these because it constitutes a very large portion of the current redis
traffic with a fair bit of that traffic writing keys that never end up
actually being read based on some recent log analysis.

Test Plan:
* Tail your development log grepping for `manage_courses`
  `tail -f log/development.log | grep manage_courses`
* Visit an account's courses page: /accounts/self
* The tailed logs should show a Redis get and setex for a permissions
  key on accounts.manage_courses
* In a console run
  `Setting.set('permissions_cache_blacklist', 'account.manage_courses')`
* Restart your web server
* Visit the account courses page
* The logs should no longer include the Redis traffic

Change-Id: I566f58e70687c4fcffe7369b94f38e62f1318ce8
Reviewed-on: https://gerrit.instructure.com/118378
Tested-by: Jenkins
Reviewed-by: Brent Burgoyne <bburgoyne@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Tucker McKnight <tmcknight@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
2017-07-12 17:52:43 +00:00
Cody Cutrer 150fcd3ca4 add more logging to passenger main thread connections
refs CNVS-37760

Change-Id: I5cd9c837ee1d8a64623486004ad04dfdf1a473d1
Reviewed-on: https://gerrit.instructure.com/118576
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-07-11 21:38:10 +00:00
Cameron Matheson f389c95e1d add graphql inflection
Change-Id: I75c2e3431a93cde1e707510455aca1fc2ac98e2b
Reviewed-on: https://gerrit.instructure.com/117046
Tested-by: Jenkins
Reviewed-by: Jonathan Featherstone <jfeatherstone@instructure.com>
QA-Review: Jonathan Featherstone <jfeatherstone@instructure.com>
Product-Review: Cameron Matheson <cameron@instructure.com>
2017-07-07 22:12:54 +00:00
Spencer Olson 0f3f341e72 add seconds_late_override to submissions table
Remove accepted_at and add seconds_late_override to the submissions
table.

closes CNVS-37893

Test Plan:
1. Verify running the migrations in this commit removes accepted_at
   and adds seconds_late_override to the submissions table.
2. Verify rolling back the migrations in this commit adds accepted_at
   and removes seconds_late_override from the submissions table.
3. Run the migration before following the next steps.
4. In a rails console, find a submission and set its late_policy_status
   to nil.

     s = Submission.find(<submission id>)
     s.update!(late_policy_status: nil)

5. Verify you cannot set seconds_late_override.

     s.update!(seconds_late_override: 60)
     s.seconds_late_override # => nil

6. Set the late_policy_status to "late" and verify you can set the
   seconds_late_override. Also verify the seconds_late method now
   returns the overridden value.

     s.update!(late_policy_status: "late", seconds_late_override: 60)
     s.seconds_late_override # => 60
     s.seconds_late # => 60
     s.update!(seconds_late_override: 20)
     s.seconds_late_override # => 20
     s.seconds_late # => 20

7. Set the late_policy_status to something other than "late" and verify
   the seconds_late_override is set to nil.

     s.update!(late_policy_status: "missing")
     s.seconds_late_override # nil

8. As a teacher, hit the submissions update API endpoint
   (PUT "/api/v1/courses/#{course.id}/assignments/#{assignment.id}/
         submissions/#{student.id}.json")

   and pass:

   submission: {
     late_policy_status: 'late',
     seconds_late_override: 60
   }

   a. Make sure the json response includes a late_policy_status of
      "late" and a seconds_late of 60.
   b. Load the submission in rails console and make sure its
      late_policy_status is set to "late" and its seconds_late_override
      is set to 60.

9. As a teacher, hit the submissions update API endpoint
   (PUT "/api/v1/courses/#{course.id}/assignments/#{assignment.id}/
         submissions/#{student.id}.json")

   and pass:

   submission: {
     late_policy_status: 'missing',
     seconds_late_override: 60
   }

   a. Make sure the json response includes a late_policy_status of
      "missing" and a seconds_late of 0.
   b. Load the submission in rails console and make sure its
      late_policy_status is set to "missing" and its
      seconds_late_override is set to nil.

Change-Id: I9a8070445482b7029454cdfbbcc266aaab5a3e4b
Reviewed-on: https://gerrit.instructure.com/117476
Tested-by: Jenkins
Reviewed-by: Matt Taylor <mtaylor@instructure.com>
Reviewed-by: Jeremy Neander <jneander@instructure.com>
Reviewed-by: Shahbaz Javeed <sjaveed@instructure.com>
QA-Review: Anju Reddy <areddy@instructure.com>
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Keith T. Garner <kgarner@instructure.com>
2017-07-07 20:57:18 +00:00
James Williams 790d93b7f6 add thread logging around connection leasing errors
refs #CNVS-37760

Change-Id: If28e1c1c9f79ce1b7cbe57f0cd24e61c68a31961
Reviewed-on: https://gerrit.instructure.com/117543
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: James Williams  <jamesw@instructure.com>
QA-Review: James Williams  <jamesw@instructure.com>
2017-06-29 20:53:13 +00:00
Cody Cutrer 75c7538888 drop rails 4.2
also, allow rails 5.1

Change-Id: If480c2a2c01a7578552dc70f4bdcf3a46df40e7b
Reviewed-on: https://gerrit.instructure.com/115654
Tested-by: Jenkins
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
2017-06-27 15:29:51 +00:00
Cody Cutrer edab6c7e46 fix DatabaseServer#run_in_each_region
we only want to still include the current region, not every single
database

Change-Id: I8a03124b3996ed20bcef55709d31891d9d37c8ec
Reviewed-on: https://gerrit.instructure.com/116941
Tested-by: Jenkins
Reviewed-by: Rob Orton <rob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-06-26 17:12:00 +00:00
Cody Cutrer f79abd65e0 use real STI for Profile model
Change-Id: Ic72ed90ba23355977b7debe5d02941b7743bad6a
Reviewed-on: https://gerrit.instructure.com/115751
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
Reviewed-by: Jon Jensen <jon@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
2017-06-19 19:53:24 +00:00
Simon Williams 99f680cb8f support update_all with join and limit
test plan:
- ContentExport.joins(:attachment).where(id: 1).limit(1).
    update_all(updated_at: Time.now.utc) should do the right thing

Change-Id: Ibffb448205a823df23a81dae6200db8cd31baf7d
Reviewed-on: https://gerrit.instructure.com/115785
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
2017-06-15 20:41:21 +00:00
Cody Cutrer a7db4d4daf rails 5: fix rails 4/5 intentional cache sharing
fixes CNVS-37606

Change-Id: I98a78f3fb13b7af013ce72586734a2f9f954d56b
Reviewed-on: https://gerrit.instructure.com/115747
Tested-by: Jenkins
Reviewed-by: Tyler Pickett <tpickett@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2017-06-15 19:25:16 +00:00