if a user crosslists via the ui, we don't want the SIS to blow away those
changes. fixes#4840, refs #4815
Change-Id: Ia3f844b3a33d9c9a6e9433dc79ce74e433f1f389
Reviewed-on: https://gerrit.instructure.com/4162
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
In particular, @batch.finish is necessary for the "blocked" job to finish.
Also, don't use send_later_if_production, just do it everywhere. By default
it won't get here anyway, so it's okay for tests.
Change-Id: I53bb4276769af481e3787073ddebb7ea5e5a5ed1
Reviewed-on: https://gerrit.instructure.com/3769
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
For "small" imports (<5000 users), this improves wall-clock time
only a bit, but is much much nicer to the db. The larger the
import, though, the more the improvement here.
Be careful that we only use transactions on things that wouldn't
block other imports running in parallel, or even progress updates
running on the same thread.
Also be smarter about how often we update progress, since that
transaction can have a considerable impact on wall clock time.
refs #3752
Change-Id: I8a79bc530f5433858ccf7d62d33b98ba25e98ffa
Reviewed-on: https://gerrit.instructure.com/3613
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
* Controlled by plugin settings; default 1
* Will re-balance incoming CSVs into at least <parallelism> new CSVs
(per import type)
* If there is a single CSV, it will be split optimally
* If there are multiple CSVs, no such guarantees are made
* Each CSV will be processed as a separate DelayedJob
* Accounts are not parallelized
* All other import types are parallelized within their own type. I.e.
all courses will be imported before any sections are started
Change-Id: Ifa6fe4e63812be95db999bdf963ab7cc1b15b75d
Reviewed-on: https://gerrit.instructure.com/3582
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: JT Olds <jt@instructure.com>
refs #4487
This consolidates our ErrorReport class with our ErrorLogging mechanism,
it's all in ErrorReport now and you call ErrorReport.log_error or
ErrorReport.log_exception to both create an ErrorReport object, and call
the hooks similar to what ErrorLogging did so that plugins for other
error handling mechanisms can be injected.
ErrorReport has a category field now, similar to how ErrorLogging used
to take a type. the /error_reports UI can filter by category.
The plugin interface was designed with Hoptoad integration in mind, but
it should be pretty general.
Change-Id: I59f7a0d44cf4b6215ad13ff92d30e1d1af607b74
Reviewed-on: https://gerrit.instructure.com/3577
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
it is not a suitable unique identifier, as evidenced by actual data
Change-Id: Ib2b2f6d523b2ad052b4eff9a35c10a6e62d394c5
Reviewed-on: https://gerrit.instructure.com/3565
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
refs #3752
also avoid an account lookup if possible
Change-Id: I8ea6b276cfdcfa86c5c70bd0051bca62737d7d77
Reviewed-on: https://gerrit.instructure.com/3564
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Zach Wily <zach@instructure.com>
* Only save the pseudonym once
* Avoid saving the user and pseudonym if nothing actually changed
* Related to that, since we skipped updating sis_batch_id
individually, we still need to do that, so do it en-masse instead.
updated_at will *not* get updated now.
These changes make re-importing the same data 3x faster than the
original import (it used to be less then 2x as fast as the original
import).
Change-Id: Ia6ea32cdeb2f4aebe041d8a51b9efe6c89bc819c
Reviewed-on: https://gerrit.instructure.com/3558
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
And avoid broadcast_policy callbacks for them
(save_without_broadcasting only affects the object you call it on,
and not everything that gets automatically saved along with it).
Change-Id: Iaeff7c038bfb2516449eaa12c8b87a1256374ca5
Reviewed-on: https://gerrit.instructure.com/3533
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
* Pre-set objects we already have that get used later
* Compare ids instead of objects to avoid loading stuff we don't
have (and don't need)
Change-Id: Idf7b701a5ae294c973b3c058dfa4d4f4d0b752bc
Reviewed-on: https://gerrit.instructure.com/3527
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Brian Whitmer <brian@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
First of all, be smarter about when we do it - it's only necessary on new
records or when something in the enrollment changes that could affect account
associations.
Second, during SIS import, gather up each user that will actually need their
associations updated and do it at the end. Normally a user will get several
enrollments during a SIS update and this can reduce the amount of work we do
by a lot.
Change-Id: I2f095dd9e7e148229e52e6b5e8b6624dfe389c31
Reviewed-on: https://gerrit.instructure.com/3200
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Hudson <hudson@instructure.com>
before we were recording these values but not doing
anything about it. This commit starts scheduling
delayed_jobs to activate/conclude enrollments
in the background when dates pass.
it does not conclude/publish courses/sections
automatically, though. if we want that to be
automated based on dates then I think we should
do that in a separate commit.
fixes#3356
Change-Id: Id94356fbc5b82196dd041fdb250607a7633cee9f
Reviewed-on: https://gerrit.instructure.com/2431
Tested-by: Hudson <hudson@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
JT Olds