closes INTEROP-6817
flag=none
Why:
- When a user's primary email changes, if the root account is set up to
use email as the login attribute, mapping which maps the user to the
Microsoft 365 user will be out-of-date.
- We never delete UserMappings except when changing Microsoft Sync
settings at an account level so currently this is almost impossible
for customers to rectify
I didn't want to actually immediately delete a mapping, because if a
CommunicationChannel changes in the middle of a sync, the missing
UserMapping could cause us to remove the user from the Microsoft 365
group, even if the user's primary email has not even really changed (for
instance, if the user added a non-primary email). So instead I use a
needs_updating field.
Test plan:
- Make sure you have an account set up with Microsoft Sync using email
as the login attribute.
- Make sure you have a course set up with Microsoft Sync, ideally with
2+ users. Make sure it is successfully synced so that there are 2+
users in the group visible in Microsoft 365 admin console.
- Look at one of the users who has a account in the Microsoft 365
tenant and change their primary email address to another email address
of a another user on the Microsoft 365 side
- Manually sync the course
(course.microsoft_sync_group.syncer_job.run_synchronously)
- Look in the Microsoft 365 admin console and check that the old user
has been removed and the new user has been added to the group.
- Change the email address again to something not corresponding to any
user on the Microsoft side
- Sync again and see that the user has been removed from the Microsoft
group
- Make sure one of your tests used the API to update the email address
- add a communication channel with the POST API endpoint, e.g.:
tok http://canvas-lms.docker/api/v1/users/123/communication_channels \
communication_channel[type]=email \
communication_channel[address]=someaddress@example.com \
skip_confirmation=true
- remove the old communication channel
tok http://canvas-lms.docker/api/v1/users/123/communication_channels
tok delete http://canvas-lms.docker/api/v1/users/123/communication_channels/456
- Repeat the steps above but update a primary email address from a
SIS import. You should be able to do this by creating a users.csv file
like this and uploading with "SIS Import" in the account menu (and
choose the "Override UI changes" checkbox):
user_id,login_id,first_name,last_name,email,status
sisuser1,sisuser1login,FirstNameOne,LastNameOne,myemailaddress@instructure.com,active
- from a console, set the needs_updating on user's UserMapping to false,
then destroy one of the user's email communication channels with
commchannel.destroy_permanently!
- make sure the UserMapping has gotten needs_updating set to true
- destroy all email communication channels for a user, or make all their
communication channels something without a user on the Microsoft 365
side. Sync the course. The user should be removed from the course
(note that you cannot remove the last course owner), and the
UserMapping should no longer exist at all.
Change-Id: I9e2f3d5e439bebc8c9eaf89b1595908213dc1981
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/294820
Product-Review: Alexis Nast <alexis.nast@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Migration-Review: Ben Rinaca <brinaca@instructure.com>
Reviewed-by: Tucker Mcknight <tmcknight@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
closes INTEROP-7451
flag=none
If the Microsoft 365 Group has been deleted on the Microsoft side, as
can happen if an admin logs in and deletes the group we create, we
shouldn't save an error report. This is an "expected" / "graceful
cancel" error, because we know we can get into this state. This normally
only happens in the partial sync, because in the full sync we recreate
the group.
To be on the safe side, I am still retrying the partial sync when we get
the 404. This required me to change the StateMachineJob code to handle a
Retry object with GracefulCancelError so it will not treat it as an
unexpected failure the final time.
Test plan:
- For ease of testing set:
Setting.set('microsoft_group_enrollments_syncing_debounce_minutes', '0')
This gets rid of the delay between adding the enrollment and the job
starting. In addition, if you want, you can change the last value of
STANDARD_RETRY_DELAY in lib/microsoft_sync/syncer_steps.rb from 300 to
something shorter.
- Have a course successfully synced with Microsoft Sync
- In Microsoft's admin UI, delete the group
- If using email as the sync login attribute state, make sure you have a
user (that can be added to the course) that has an an email that has
already been confirmed (the CommunicationChannel's workflow_state
should be "active")
- Add such a user (that has a corresponding user on the Microsoft side)
to a course, and update the enrollment to 'active':
e = Enrollment.last # etc.
e.update workflow_state: 'active'
The job will only try to add the user after the enrollment is active.
- Make sure you have jobs running. The job should run and be retried
three times. After one of the intermediate failues, check the
MicrosoftSync::Group. The workflow_state should be `retrying` and the
job_state should mention a GroupNotFoundGracefulCancelError.
- After the last failure, the MicrosoftSync::Group workflow_state should
be `errored`. In the UI, in the Course settings -> Integrations, you
should see the message "The Microsoft 365 Group created by sync no
longer exists..."
- Run a full sync
(course.microsoft_sync_group.syncer_job.run_synchronously) to recreate
the group. Add another user as before (or remove the user, run a full
sync, and add the same user back) and let a partial sync run. Make
sure the partial sync runs successfully and adds the user to the
Microsoft 365 Group.
- Set the debounce setting back to its original value:
Setting.set('microsoft_group_enrollments_syncing_debounce_minutes', '10')
Change-Id: I27b906655255ec5a3384313c529aedb48acd258b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/293325
Reviewed-by: Tucker Mcknight <tmcknight@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Alexis Nast <alexis.nast@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Currently, when we look up email addresses for a user, we just look on
the shard the sync is happening on (the shard the course is on).
CommunicationChannels almost live on the user's home shard, so if there
are cross-shard users, we need to activate their shards before looking
for the CommunicationChannel.
I also modified a spec helper because it was not detecting an error's
call to I18n.t correctly for some reason for my local MRA setup. (I
checked that the new helper also fails when the error class doesn't
actually call I18n.t)
closes INTEROP-7147
flag=none
Test plan:
- Have MRA Canvas set up
- Ensure you have the Microsoft Sync creds set in
config/dynamic_settings.yml
- One on shard, set up Microsoft Sync in the root account and in a
course.
- turn on feature flag
- in account settings enable sync with our test tenant and use "email"
and "UPN" as the local and remote lookup fields
- in Course Settings -> Integrations turn on sync for the course
- Add two users to the course, one in another root account. At least one
should be a teacher. Both should have emails corresponding to users in
our Microsoft test tenant. Set the enrollments' workflow_state to
"active".
- Make communication channels for each users in their respective shards.
Make sure each shard has only the communication channel for the user
local to that shard, i.e., there are no communication channels with
user_id > Shard::IDS_PER_SHARD. Confirm the communication channels:
run `communication_channbel.confirm` on each, which should set the
workflow_state to "active".
- Sync the course
course.microsoft_sync_group.syncer_job.run_synchronously
- The sync should succeed and the users should be visible in the
Microsoft admin console
Change-Id: Ic87be3d9dfe0578370f2f04e8509361b75269961
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283233
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Reviewed-by: Mysti Lilla <mysti@instructure.com>
closes INTEROP-7232
flag=none
Prevents a partial sync from masking the error in the UI and making the
user think the sync is working great, when in reality, some users may
have not been added and Team may have not even been created for the
Group.
This solution is more complicated than I would like but I really think
this is the simplest way of avoiding the problem. See the ticket for
discussion and another discarded approach.
I also changed COMPLETE and IGNORE to have their own classes so it
makes the debugging show "finished step with Ignore" instead of
"finished step with Object".
Test plan:
- enable Microsoft Sync in course settings for a course where there
are teachers but they don't exist on the Microsoft side.
- kick off a full sync, e.g. from a console:
group.syncer_job.run_synchronously
- observe that it fails. Look in the UI (course settings ->
integrations) and check that it shows the the correct error,
saying that "no users corresponding to the instructors of the Canvas
course could be found on the Microsoft side."
- add a user to the course -- that *does* exist on the MSFT -- to the
course as a teacher. Set the Enrollment's workflow_state, and (if
necessary) the user's CommunicationChannel's workflow_state, to
"active"
- wait 10 minutes until a partial sync job runs (if you have jobs
running) or just kick off a partial sync with:
group.reload.syncer_job.run_synchronously(:partial)
- from the logs / output you can check that that didn't actually hit
the MS API, and that the step returned with Ignore
- check in the UI to see that the same error is still there
- kick off another full sync. this should run successfully now that
there is a teacher that does exist on the MS side. in the MS admin
console check that the group has the use. Also you can see that a team
has been created for the group: if the Group info sidebar, next to
"Delete Team" under the name of the Group, will be a link "Open in
Teams")
- Now that the error has been cleared, add another user and run partial
sync to make sure that works and it adds the new user
Change-Id: I760cf7a69f52d48866cb11511c30739de119c0dd
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282798
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
auto-corrected, with post-review to look for things that are only
hash-like (one found; fixed by calling `to_h` beforehand)
Change-Id: I1a1f273513ff466197a4da6a7b6f68565a08abe0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278500
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Previously, we never deleted the mappings between a Canvas User and
Microsoft user AAD object ID after caching it. So if a Microsoft admin
deleted users from Microsoft 365, some courses would fail to sync
because the AAD object ID we had didn't exist on the Microsoft side, and
ge would never try to re-look it up.
Now, if we try to add a user and the user's AAD object ID doesn't exist
on the Microsoft side, we will notice that and delete the mapping. The
next time a sync is run, we will successfully re-lookup the Canvas user
and add them if a different Microsoft user exists for them.
If we are trying to remove a user from a group and the user doesn't
exist, we will still error, as Microsoft's API doesn't differentiate
between the user not existing, group not existing, or the user just not
being in the group. However this case is not as bad as it can only
happen in a partial sync (since in a full sync we get the full list of
users in the group from the API, and that list will not include deleted
users). So if that happened, a full sync would fix the course/MS group
and allow it to sync again.
This commit:
* Modifies add_users_to_group (patch api -- all users at once) to notice
when the API sends back a 404 that some user does not exist. At this
point we only know that at least one user doesn't exist, so we use the
batch API (all users as separate requests in a batch) to get a list of
all such users.
* Changes add_users_to_group_via_batch to notice when the API returns a
404 saying the user does not exist. To communicate this up, we have
a new method nonexistent_users on GroupMembershipChangeResult
* Changes special_case.rb to support a block tester. Also use an
encapsulate response which can contain a `batch_request_id` for batch
requests. These are used by the new code.
closes INTEROP-7000
Test plan:
* have a course with some users (enrollments must be 'active',
CommunicationChannels must be 'active' if email is used for lookup
key) and sync with Microsoft.
* delete users on MS side
* run a full sync (e.g. group.syncer_job.run_synchronously). sync should
succeed, the usermapping for the user that no longer has a user on the
Microsoft side should be deleted.
* delete the group on MS side.
* add a user (active enrollment, active communicationchannel as before)
with a user on the microsoft side and run a full sync. that should
should recreate the group on the microsoft side (as before)
* delete a user on the MS side and then immediately remove the user from
the course on the Canvas side. run a partial sync (e.g.
group.syncer_job.run_synchronously(:partial)). That should be ignored
(as before) because we just assume the user is not in the group (we
can't distinguish between that and the user not existing when removing
users).
Change-Id: I5230b32d7cbfd1462177e3e597ba7d9633f055d6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278068
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
In the next commit I need add_users_ignore_duplicates to return more
complicate information than just the list of users returned.
Encapsulating that information in an new class instead of returning
a complicate array-hash structure seemed the most clear thing to do.
refs INTEROP-7000
Test plan:
- TODO, should smoke test all previous commits
Change-Id: I65e5ccbbc49ff3eaee8fcdb5e15158a95e4cf133
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277726
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Finish splitting up GraphService -- remove delegates and call the
methods in the GroupsEndpoints class directly, and rename the methods to
not be redundant.
refs INTEROP-7000
Test plan:
- specs. we can do some overall graph service smoke tests after all my
refactoring
Change-Id: Ia5856e11f4f375da6f7feb353ce144a001a78aee
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277695
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Remove delegates and call the methods in the *Endpoints classes
directly, for everything but GroupsEndpoints. Renames methods to not be
redundant.
refs INTEROP-7000
Test plan:
- specs. we can do some overall graph service smoke tests after all my
refactoring
Change-Id: I13463e6611abfbcf6c33838e8c28b1b97953a123
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/278080
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
This makes classes for the different endpoints and for now delegates
calls to them.
refs INTEROP-7000
Test plan:
- specs. we can do some overall graph service smoke tests after all my
refactoring
Change-Id: I0bc81bae35066c5b94c96e37d8b0de9dbe2e821f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277691
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
This method is unused except for rare debugging and is not that hard to
reproduce when needed. This file is big and complicated enough already.
Test plan:
- specs. we can do some overall graph service smoke tests after all my
refactoring
Change-Id: I525b5a39b45612eea85106e676ec7917822cd64f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277690
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
preparation for future commits.
refs INTEROP-7000
Test plan:
- specs. we can do some overall graph service smoke tests after all my
refactoring
Change-Id: Ibca941f1fa1e3efa6dd6e844b7a4d1787b2701bc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277689
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
[skip-stages=Flakey]
auto-corrected, with manual review to identify possible nilness
Change-Id: I205436e5c3cb37aae99ea552c7d14e6d1a04ef06
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277893
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[skip-stages=Flakey]
the balance. mostly. Lint/UriEscapeUnescape is put in the pending
block because it's so touchy, and I didn't want to deal with it
right now
all manual
Change-Id: Ibeb81e013f56f160d51f7d237a9bcfe98daa1e53
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277569
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Test plan:
- go to course settings integrations page
- make sure copy mentioning max number of users in a team still shows
correct numbers (25,000 and 100)
refs INTEROP-6805
flag=microsoft_group_enrollments_syncing
Change-Id: I01c6839bd63d8f221159050a68c04b28da09fda8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276585
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
This was discovered in QA for a previous commit. If a course has over
100 owners, the sync fails, and then we run a partial sync, we can run
into the case where the partial sync tries to add 20 owners but many if
not all or duplicates. If there are >= 81 owners, Microsoft won't
realize this but instead say "there are too many users". Instead of
trusting this message, we should fall back to the batch API -- because
it adds each user individually, it will ignore any that are already in
the group instead of saying there isn't room.
Test plan:
- Have a Microsoft group with >= 82 owners
- From a rails console, get the graph service object and then get a list
of 20 owners in the group.
def all_owners_ids(gs, group_id)
[].tap do |list|
gs.list_group_owners(group_id) do |slice|
list.concat slice.map{|u| u['id']}
end
end
end
group = MicrosoftSync::Group.first
gs = group.syncer_job.steps_object.send:graph_service
owners = all_owners_ids(gs, group.ms_group_id).take(20)
gid = group.ms_group_id
- Remove one of the owners from the group:
one = [owners.last]
gs.remove_group_users_ignore_missing(gid, owners: [owners.last])
- Wait a couple seconds (API is eventually consistent) and check that
the user is no longer in the group:
all_owners_ids(gs, gid).include?(owners.last)
- Add the 20 owners -- 19 of which should already be in the group:
gs.add_users_to_group_ignore_duplicates(gid, owners: owners)
- The function should return a list of the 19 duplicates. You should see
it log that it first tried the 'patch' endpoint and fell back tothe
'post $batch' endpoint.
- Wait a couple seconds and check that the user was added back in.
gs.list_group_owners(gid).map{|g| g['id']}.include?(owners.last)
refs INTEROP-6805
flag=microsoft_group_enrollments_syncing
Change-Id: I4a3dd47edb62b29ba8278182b37894b28604ceda
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276318
Reviewed-by: Sean Scally <sean.scally@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Test plan:
- have a course with > 100 owners
- try to enable sync on it
- check copy is updated in the two places
refs INTEROP-6805
flag=microsoft_group_enrollments_syncing
Change-Id: I911c6ca6998918a3a49cbbbb2473566800ea2d60
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276483
Product-Review: Karl Lloyd <karl@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs INTEROP-6805
flag=microsoft_group_enrollments_syncing
Test plan:
- have a course with one owner
- remove an owner and add a different owner. recall that enrollments
have to be in an "active" workflow_state to count.
- run a partial sync (`group.syncer_job.run_synchronously :partial`)
- sync should fail, but it should still add the new owner. Check in the
Microsoft admin console. Both owners will exist in the group.
- run a full sync (`group.syncer_job.run_synchronously`). Check in the
MS admin console. The old owner should now be removed.
- Repeat but run a full sync instead of a partial sync the first time.
Change-Id: I0cce5a50a51f40ed36f5eea928638c4782b31937
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276019
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
Also fix default special_cases ([] instead of {}) in graph_service/http.rb
The diff http.rb doesn't show nicely here, try:
diff --ignore-space-change \
<(git show HEAD^:lib/microsoft_sync/graph_service_http.rb) \
<(git show HEAD:lib/microsoft_sync/graph_service/http.rb)
refs INTEROP-6805
flag=microsoft_group_enrollments_syncing
Test plan:
- run through a sync, make sure it still works
Change-Id: I12d4339c0c1a50417ed23f9476ca12b1b9f711a4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275802
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
closes INTEROP-6805
flag=microsoft_group_enrollments_syncing
Test plan:
- note: this is closely based on test plan for last commit
- enable sync for a course
- add 101 teachers that have users in our test tenant. I
made a bunch of test users to use owners you can use. What I did was
generate the list of UPNs with
'echo evantest{001..101}deleteafternov2021@...' and add them to my
course)
- Remember the enrollments have to be all active and the communication
channels 'registered', so I did
`CommunicationChannel.update_all(workflow_state: 'active')`
followed by
`ms_group.course.enrollments.map{|e| e.update workflow_state: 'active'}`
which will add the users to the PartialSyncChanges table.
- run a partial sync (e.g.
`group.syncer_job.run_synchronously :partial`)
- in the course settings check the Microsoft Sync integration, it should
be disabled. Note that it won't show any error because errors are not
shown when the sync is disabled, but trying to enable it will show the
error shown when trying to enable a course with too many enrollments
(added in a previous commit)
- in a console, check that last_error is set on the group
- remove the 'check_for_enrollment_limits' before_action hook in
app/controllers/microsoft_sync/groups_controller.rb
- also remove the 'rescue Errors::OwnersQuotaExceeded' block on
lib/microsoft_sync/syncer_steps.rb:263 so we can be sure we are testing
the step_full_sync_prerequisites code path
- enable the sync again
- run a full sync
- check the course settings page again. Again the sync should be
disabled. And last_error should should have the "too many teachers"
error message
Change-Id: Ied4faf7f997040e224c1cbdcdb7f0c5d24c4e02e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274953
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
* This needs to be done when adding users, either via the endpoint which
adds multiple users, or adding via the "$batch" endpoint (which batch
subrequests which each add one user).
* To ease handling of the ever-growing number of special cases, I
refactored code away from using the "special case block" paradigm to a
"special_cases" hash with matchers (status code + body regex)
More work to rescue the exception and disable sync will be done in
the next commit.
flag=microsoft_group_enrollments_syncing
refs INTEROP-6805
Test plan:
- have a course setup with Microsoft Sync enabled and run a full sync
once.
- add 101 teachers that have users in our test tenant. I
made a bunch of test users to use owners you can use. What I did was
generate the list of UPNs with
'echo evantest{001..101}deleteafternov2021@...' and add them to my
course)
- Remember the enrollments have to be all active and the communication
channels 'registered', so I did
`CommunicationChannel.update_all(workflow_state: 'registered')`
followed by
`ms_group.course.enrollments.map{|e| e.update workflow_state: 'active'}`,
which will add the users to the PartialSyncChanges table.
- run a partial sync (e.g.
`group.syncer_job.run_synchronously :partial`) It should fail with the
Errors::OwnersQuotaExceeded error, coming from the request() call in
add_users_to_group_ignore_duplicates() (not from
add_users_to_group_via_batch())
- run:
MicrosoftSync::UserMapping.to_a.each_slice(20) do |users|
graph_service.add_users_to_group_via_batch(
group.ms_group_id, [], users.map(&:aad_id)
)
end
That should also fail with the same error.
- Note: I didn't add 25,000 members to see the too many members error,
but I assume it looks nearly the same.
Change-Id: I425b7c8f09609db11fed31a475ffac2f77b1d3e7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274798
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
flag=microsoft_group_enrollments_syncing
refs INTEROP-6805
Test plan:
- modify limits in membership_diff.rb
- for each limit, have a course with more owners or users than the
limits. The users' enrollments must all have a workflow_state of
"active" or "creation_pending"
- in the UI try to enable sync for the course
- it should fail and give an error
- make sure you can enable and disable sync for courses below the limits
Change-Id: I7def5f664dc57e640529bf6a8b3b3d12c16556e4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273810
Reviewed-by: Mysti Lilla <mysti@instructure.com>
QA-Review: Mysti Lilla <mysti@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Karl Lloyd <karl@instructure.com>
Only "active" and "creation_pending" enrollments should be synced to
Microsoft.
closes INTEROP-7030
flag=microsoft_group_enrollments_syncing
- for ease of testing set:
Setting.set('microsoft_group_enrollments_syncing_debounce_minutes', '0')
- Have a course with at least one teacher enrolled with the enrollment
in an "active" or "creation_pending" state.
- Start a jobs container
- Add an enrollment (for a user which has and check it is in the
"invited" state)
- After the partial sync job runs, check that the user is not in the
group on the Microsoft side. You can use the MS admin console, or
group.syncer_job.steps_object.send(:graph_service_helpers)
.get_group_users_aad_ids(group.ms_group_id)
- Manually kick off a full sync (e.g. group.syncer_job.run_later) and
check that the user has still not been added.
- Login as the user and accept the invitation. (You can just masquerade
as the user as long as user has at least one login)
- Check that a partial sync job runs and then adds the user to the
group.
- Manually kick off a full sync and make sure the user is still part of
the group.
- Deactivate the user's subscription (choose the somewhat-deceptively
named "Deactivate User" in the course's People page)
- Check that a partial sync job runs and removes the user from the
group.
- Manually kick off a full sync and make sure the user is still not part
of the group.
- Change back the cooldown period if you want:
Setting.set('microsoft_group_enrollments_syncing_debounce_minutes', '10')
Change-Id: Ie7f70c0946cce80b1922d793646a41682b3ee2c0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272436
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
If a job goes into a retry state (i.e., a job is scheduled in the
future to start off from an intermediate step), and during this
time the user deletes and restores a Group before the job continues,
when the job starts again, it will be confused because the Group
is not deleted but the job_state has been cleared. (In a continuing
job, the job_state holds the current step, which must match up
with the argument in the job).
The solution is just to add {restored: true} to the job state to signal
that to an in-progress job (of which there can only be one) that it can
be ignored. When the continuing job starts, it will understand that
the group has been deleted and restored since the job was started,
so it can be dropped.
Note that if we didn't clear the job_state at all when
deleting/retrying, we could end up in a similarly confusing situation
such that kicking off a new job after restoring would be confused
because it's not in a fresh (no job_state) state.
closes INTEROP-7019
flag=microsoft_group_enrollments_syncing
Test plan:
- increase DELAY_BEFORE_UPDATE_GROUP in SyncerSteps to 60 seconds or so
to facilitate testing, so the job will be in a "retrying" state for
longer.
- run a jobs container
- Clear ms_group_id on the group and kick off a sync job
(group.syncer_job.run_later)
- while the job is waiting [retrying] (you can double check by checking
the group workflow_state) delete and restore the group in the UI.
- when the job continues there should not be an InternalError. Instead
the message "In-progress job starting again but job state record was
deleted & restored since" should be shown. The job should not
continue, but rather be dropped.
- check the state of the MicrosoftSync::Group. It should still be in
"pending".
Change-Id: Iac0e997e4b28a526262153919c562fd3c3be3dde
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271943
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
On retry, JSON body was being JSON-ified twice, because the
request_without_metrics method mutated the hash.
Closes INTEROP-7020
flag=microsoft_group_enrollments_syncing
Test plan:
- Run these steps first to make sure you've repro'd the bug:
- Add the following in request() right before the "if
check_for_expected_response..." line (111)
unless (tmp_tries ||= 0) > 0
tmp_tries += 1
raise Timeout::Error, 'fake'
end
- Run:
gs = group.syncer_job.steps_object.send(:graph_service)
gs.create_education_class_team("abc")
- Before this change, Microsoft will return a 400 with
"Empty Payload. JSON content expected."
- Repeat the steps on the this commit. Instead you will get the proper
message (groupId needs to be a valid GUID)
Change-Id: I21a8000637f1d4c0ffc966fc7666daef11d4c26b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272072
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Michael Ziwisky <mziwisky@instructure.com>
Product-Review: Michael Ziwisky <mziwisky@instructure.com>
It's preferable to load account_settings (and possibly other values)
every time a job runs so we don't get old values. Previously, these were
being serialized in the JSON in continuing jobs (jobs retried or
delayed using DelayedNextStep).
closes INTEROP-7017
flag=microsoft_group_enrollments_syncing
Test plan:
- check out the commit before this one.
- from a console, run:
group = MicrosoftSync::Group.last
sj = group.syncer_job
sj.steps_object.send(:account_settings)
sj.run_later
- See from the output that the job's handler has "account_settings" in
it, under steps_object (for better indentation you can also puts
Delayed::Job.last.handler)
- Do the same on this commit
- The steps_object in the handler should have only a "group"
- Run a full job asynchronously (run a jobs container. then run
group.syncer_job.run_later) and make sure it correctly runs and
creates/updates a Microsoft group
Change-Id: I9c3193e6aff4fade7094815dba7d2740f2096cec
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271938
Reviewed-by: Michael Ziwisky <mziwisky@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Psych has safe_load now, and it's fairly trivial to convert our existing
overrides to use that instead
Change-Id: I2648df8d4574e15fc9072a25882e318d902765c3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271939
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Some users have tried to use a tenant that is not a Microsoft "Education
tenant". Right now this is counted as an unexpected error in our stats.
This treats it as expected and shows the user a descriptive message
rather than just "Microsoft API returned 400"
flag=microsoft_group_enrollments_syncing
closes INTEROP-6993
Test plan:
- Run a sync and make sure it still works. (The first step will be to
check if the Microsoft group exists with the GET /education/classes
endpoint I changed)
Change-Id: I2b866bd8e3188f59d674dbebc164031b6f5fe9a7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271267
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
I see a case of the user removing the last owner. This causes a generic
BatchRequestFailed error currently, which will confuse the user, but
more importantly clog up error logs / Sentry with something which is
expected.
We always add new owners before removing old ones in execute_diff so if
the user adds a different owner the sync will succeed.
flag=microsoft_group_enrollments_syncing
refs INTEROP-6970
Test plan:
- have a course with one owner that is synced
- remove last owner in a course. Note: if you have multiple owners and
remove them in one sync, you will likely not get an error, because the
MS API is eventually-consistent and we remove all at the same time and
it won't know that you have removed all owners
- run a partial sync, e.g. g.syncer_job.run_synchronously(:partial)
- note in log output that there is no big message about making a Canvas
ErrorReport
- check errors in Course Settings -> Integrations. You should see the
friendly message about the course having no owners
- add a new owner and re-run partial sync
- it should succeed
Change-Id: I3aad866a5f0e11234ae1402fc7ae401ce566e609
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271148
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
The Microsoft API returns a 400 or other error code when the description
is too long in some endpoints. These limits were determined
experimentally.
Other fields, namely `name` and `sis_source_id`, do not have this
problem as they are already limited to 255 characters in the Canvas
database.
flag=microsoft_group_enrollments_syncing
refs INTEROP-6970
Test plan:
- Create a course with a description of over 1025 characters, or modify
an existing course and delete its associated MicrosoftSync::Group
- Sync the course
- It should succeed. Check the description has '...' and a length of
1024 with the following:
g = MicrosoftSync::Group.last
gs = g.syncer_job.steps_object.send(:graph_service)
ap gs.get_group(g.ms_group_id)['description']
ap gs.get_group(g.ms_group_id)['description'].length
- Also check that the lmsCourseDescription has '...' and a length of 256
fields = ['microsoft_EducationClassLmsExt']
res = gs.get_group(g.ms_group_id, select: fields)
ap res
ap res["microsoft_EducationClassLmsExt"]["lmsCourseDescription"].length
Change-Id: I7736286fbcb5345c8f19d9ac554dc644b4644199
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271107
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Added support to the accounts update_api action for two new settings
related to Microsoft Sync.
The first setting, microsoft_sync_login_attribute_suffix is the
suffix that will be appended to the result of
microsoft_sync_login_attribute for the purposes of syncing.
The second setting, microsoft_sync_remote_attribute, is the Active
Directory attribute that will be used to associate Canvas users to
Microsoft users.
For example, if you had a login attribute of sis_user_id, a suffix of
@example.com and a remote attribute of mailNickname, then a query to
Microsoft would combine a user's SIS ID with the suffix, for a result of
12345@example.com and would then query Microsoft to look for any user's
who had a mailNickname that matched this new ULUV (User LookUp Value).
Note that the Selenium tests for the UI had to be taken out, as the
current UI doesn't work with this new backend. They will be readded and
adjusted in the next commit.
In addition, added support for a new login_attribute named
integration_id. This gives schools even more flexibility when
configuring MSFT Sync.
Lastly, moved all of the settings validation code out of a concern and
into a class, which explicitly defines what dependencies the validation
code has, and also just makes more sense in general.
closes INTEROP-6921
flag = microsoft_group_enrollments_syncing
test-plan:
* You'll be sending PUT requests to /api/v1/accounts/#{account.id} where
the account.id is the id of the account you're using to test this. Use
any tool you'd like for this, like cURL or Postman.
* The body of your requests will look like this in JSON:
{
"account": {
"settings": {
<the name of each of the parameters
we're testing, such as
microsoft_sync_remote_attribute>
}
}
}
* Enable the microsoft_group_enrollments_syncing feature flag, either
through the UI or a rails console with
<account.root_account.enable_feature!(:microsoft_group_enrollments_syncing)>
* Send a request to the endpoint and set all of the fields to valid
values. The name of each field is:
microsoft_sync_enabled
microsoft_sync_tenant
microsoft_sync_login_attribute
microsoft_sync_login_attribute_suffix
microsoft_sync_remote_attribute
Enabled has to be a boolean, tenant has to be a valid domain name, the
login attribute has to be one of (email preferred_username
sis_user_id), the suffix can be anything under 255 characters without
whitespace, and the remote attribute must be one of
(userPrincipalName mail mailNickname)
This request should succeed with a 200. Send a GET request to the same
endpoint as above and ensure the settings have been saved.
* For each of the following requests, you should get a 400 with an error
message and the account settings shouldn't change.
* Send a request with a suffix over 255 characters.
* Send a request with a suffix with whitepspace in it.
* Send a request with an invalid remote attribute.
* Send a request trying to enable sync, but without specifying
any settings. You can also mix it up and specify only a few
settings. Note that the suffix is optional, so if you only
omit that setting, you won't get an error.
* Send a request to the same endpoint, this time without authorization.
It should fail with an auth required message.
* Turn the feature flag off and try and change some settings. You should
get a message saying the feature flag is off.
Change-Id: If80f278e2294515be3e0b8dbf38dd22303e016b1
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269542
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Evan Battaglia <ebattaglia@instructure.com>
QA-Review: Evan Battaglia <ebattaglia@instructure.com>
Product-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
w/o messing up state. Before, the state would be kept in a 'running'
state which we would have to remember to manually fix.
Note that run_synchronously is not used in running production code, only
for console debugging.
closes INTEROP-6724
flag=microsoft_group_enrollments_syncing
Test plan:
- In a rails console, run:
g = my_course.microsoft_sync_group
g.syncer_job.run_synchronously
- while it is running press Ctrl-C. The job should stop.
- check `g.reload`. workflow_state should be 'errored' and 'last_error'
should be a JSON blob without a public_message
Change-Id: I0a579f9eb1f4d096aa1a371815929725334765a2
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/270120
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Note that even previous to this if the job were enqueued the fake user's
username is a random hash and the SIS ID (?) and email are not present so
the user would never really get added. But it's a waste of time to kick
off a job.
Test plan:
- Have a course set up with sync
- Click "Student View" on the upper right corner of the page on the
page for the course
- Check that there is now a StudentViewEnrollment
- Check that the PartialSyncChanges and Delayed::Job tables are
unchanged and there is no job
closes INTEROP-6922
flag=microsoft_group_enrollments_syncing
Change-Id: I2e6ba915355a45875e3a24064ab01084eed650a7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269926
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
* changes GracefulCancelErrorMixin mixin to class GracefulCancelError.
All graceful cancel errors should be public, because they are expected
errors so should have a public, internationalized, message.
* a few things that had GracefulCancelErrorMixin were not PublicErrors.
Since they now are, I added public_message to them.
* Not strictly related but I made BatchRequestFailed and
BatchRequestThrottled PublicErrors.
refs INTEROP-6739
flag=microsoft_group_enrollments_syncing
Test plan:
- Try changing your tenant to something we don't have access to,
like microsoft.onmicrosoft.com
- Sync and look at the course integration settings. You should see the
nice error saying 'Application not authorized for tenant. Please make
sure your admin has granted access for us to access your Microsoft
tenant.'
- Check the MicrosoftSync::Group object, there should be a last_error
but not a last_error_report_id (Graceful Cancel Errors don't create
error reports).
Change-Id: I583e24d1c9b0f50177c32010cf39c2d853c84a2a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269418
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
Reviewed-by: Sean Scally <sean.scally@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Xander Moffatt <xmoffatt@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Errors happen in a job, so to localize the errors to whatever the user
viewing the page sees, we need to store the un-i18n-ized version in the
database. However, if we do that, the messages won't be extracted to the
i18n yml file; so we still need to call I18n.t (we do this in #public_error)
with a constant string so the extractor can pick them out.
I'm not sure if jobs are even run in a non-en locale, but in case they
are, we temporarily set the locale to 'en' before running that I18n.t()
for that error to make sure we store the un-i18n-ized version.
TODO follow up commits:
* change more errors to PublicErrors
* GracefulCancelError can be a subclass of PublicError
Test plan:
- have a course with no owners (teachers/tas/designers) and try to sync.
- check the last_error field on the MicrosoftSync::Group model. It
should be a JSON hash with appropriate public_error_message and no
public_interpolated_values.
- check the UI (course integrations settings) to make sure the error
shows up there (just the public message, not JSON)
- Add the following to es.config/locales/yml:
microsoft_365_allows_a_maximum_of_max_owners_in_a__b448f5de: "Microsoft 365 permite como máximo %{max} administradores de un Microsoft Team."
- change MembershipDiff::MAX_ENROLLMENT_OWNERS to 2. restart Canvas.
Add two owners in a course and try to sync.
- check the last_error. It should have a public_message and separate
public_interpolated_values with "max": 2.
- check the UI to make sure the error message shows up with
the interpolations.
- change your language to Español
(/profile/settings -> Edit Settings) and reload the course settings
integration page. The error message should now show up in Spanish.
¡Empieza la fiesta!
refs INTEROP-6739
flag=microsoft_group_enrollments_syncing
Change-Id: Ia723946e2d2f2b58fdaeb2a99cacfd23ed6f3197
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269064
Reviewed-by: Xander Moffatt <xmoffatt@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Also fixes a bug where users in a course that had a null value for the
login attribute (SIS id is the one case I know of for sure) would make
UsersUlvsFinder crash.
closes INTEROP-6927
flag=microsoft_group_enrollments_syncing
Test plan:
- have a course with user(s) with pseudonym(s) with an integration id
equal to the UPN (or email) of remote users in our test tenant
- set the root account micrososft_sync_login_attribute setting to
"integration_id"
- clear the UserMapping table if necessary, and run a sync for the
course
- check in the Microsoft admin console, the user(s) should be in the
team there
- on one of the users, set the "integration_id" field on the pseudonym
to null, e.g.
my_pseudonym.update integration_id: nil
- clear the UserMapping table again
- run a sync. the sync should run successfully and the user should not
be in the Microsoft team anymore. (or, if there are now no teachers
with mappings, the job will actually fail saying a group can't have
no owners)
Change-Id: I15d571a663247af4db60b55298ecbbd299a320a6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269672
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Some of our customers don't have their Microsoft accounts set up such
that the UPN (UserPrincipalName) of Microsoft users is exactly equal to
the username, SIS id, or email of the corresponding Canvas user. So,
they need another way to link Canvas users and Microsoft users. This
to the (Canvas username | SIS id | email) and then using that to match
commit makes the linking more flexible by allowing a suffix to be added
to different fields (mail or mailNickname) instead of always matching to
the UPN.
closes INTEROP-6739
flag=microsoft_group_enrollments_syncing
Test plan:
- NOTE: this is the test plan I followed, but it is quite complicated.
You can use the users I already created on our Microsoft test tenant, or you
might want to test a few simpler scenarios.
- in our Microsoft tenant set up new users with emails, mailNicknames,
and UPNs that are all new and completely different from each other.
The easiest way is to create users in the admin console (add a license
which allows them to have an email address) then use
https://developer.microsoft.com/en-us/graph/graph-explorer and the
PATCH /v1.0/users/...aadid... endpoint to set the properties using
JSON blobs for each, e.g.:
{
"displayName": "INTEROP User 1",
"userPrincipalName": "INTEROPabc@OURTESTTENANT.onmicrosoft.com",
"mail": "ebattaglia.def@OURTESTTENANT.onmicrosoft.com",
"mailNickname":"INTEROPghi"
}
{
"displayName": "INTEROP User 2",
"userPrincipalName": "ebattaglia@OURTESTTENANT.onmicrosoft.com",
"mail": "ebattaglia.mno@OURTESTTENANT.onmicrosoft.com",
"mailNickname":"INTEROPpqr"
}
{
"displayName": "INTEROP User 3",
"userPrincipalName": "INTEROPstu@OURTESTTENANT.onmicrosoft.com",
"mail": "ebattaglia1@OURTESTTENANT.onmicrosoft.com",
"mailNickname":"ebattaglia1"
}
- Make two courses, "course I" and "course II", and enable sync on them.
- Set up new Canvas users with SIS ids, emails, and usernames that are
all completely differnt but that can be related to the UPNs, mails, and
mailNicknames above in various ways, some with suffixes. e.g.:
User A: username=INTEROPabc@OURTESTTENANT.onmicrosoft.comemail=ebattaglia.mno@OURTESTTENANT.onmicrosoft.com
SISid=INTEROPgh
User B: username=ebattaglia.mno
email=ebattaglia1@OURTESTTENANT.onmicrosoft.comSISid=INTEROPstu@OURTESTTENANT.onmicrosoft.com
User C: username=ebattaglia1
email=ebattaglia.def@OURTESTTENANT.onmicrosoft.com
SISid=ebattaglia
AlwaysATeacher1: username=interop_teacher
SISid=interop_teacher
email=interop_teacher@OURTESTTENANT.onmicrosoft.com
AlwaysATeacher2: username=interop_student
SISid=interop_student
email=interop_student@OURTESTTENANT.onmicrosoft.com
The last two are just to make sure there are always some teacher
in each course (otherwise the sync will fail).
This may be easiest to do by adding new users to a course and then
editing their properties. I used
User.update_all workflow_state: 'active'
CommunicationChannel.update_all workflow_state: 'registered'
To fix up pending states (NOTE that updates all your local users, so
make sure you don't care about pending states for other users)
- Enroll:
in Course I: User A as a teacher and user B as student
in Course II: User B as a teacher and User C as student.
in both: AlwaysATeacher1 and AlwaysATeacher2 as
(For some reason looking up by username instead of email when adding
user to a course worked better for me.)
- Change the root account settings to map the users in varying ways,
e.g.
c_i = Course.find(...)
c_ii = Course.find(...)
acct = c_i.root_account
acct.settings[:microsoft_sync_login_attribute] = 'preferred_username'
acct.settings[:microsoft_sync_login_attribute_suffix] = \
'@OURTESTTENANT.onmicrosoft.com'`
acct.settings[:microsoft_sync_remote_attribute] = 'mail'
acct.save
- Clear user mappings:
MicrosoftSync::UserMapping.delete_all
- Run syncs on both courses. Check in the Micrososft admin console that
the expected users are left in the course in the proper roles.
c_i.microsoft_sync_group.syncer_job.run_synchronously
c_ii.microsoft_sync_group.syncer_job.run_synchronously
- Run for varying configuration of login_attribute, suffix,
remote_attribute.
- Examples for above setups (ignoring 'always teachers' users)
SIS id, username, and email on Canvas side as above)
1) LA=preferred_username suffix="@OURTESTTENANT.onmicrosoft.com" RA=mail
(User B maps to INTEROP User 2, User C maps to INTEROP User 3)
=> Course I: members=[INTEROP User 2]
Course II: owners=[INTEROP User 2] members=[INTEROP User 2, INTEROP User 3]
2) LA=email suffix= RA=mail
=> User A maps to INTEROP User 2, User B maps to INTEROP User 3, User C maps to INTEROP User 1
=> Course I: owners=[INTEROP User 2] members=[INTEROP User 2, INTEROP User 3]
Course II: owners=[INTEROP User 3] members=[INTEROP User 1, INTEROP User 3]
3) LA=sis_user_id suffix=i RA=mailNickname
=> User A maps to INTEROP User 1
=> Course I: owners=[INTEROP User 1] members=[INTEROP User 1] (no
'always teachers' users)
=> Course II sync will fail because group doesn't have any owners.
(it doesn't have any users at all that match on the Microsoft
side actually)
3) LA=sis_user_id suffix= RA=UPN
(User B maps to INTEROP User 3)
=> Course I: members=[INTEROP User 3]
Course II: owners=[INTEROP User 3] members=[INTEROP User 3]
4) LA=sis_user_id suffix= RA=nil (default = UPN)
Same as above
- Choose one of the above setups and run partial syncs by
adding/removing users from one of the courses and checking that the
users are added/removed correcly. Clear UserMappings before each
partial sync run.
Change-Id: If10020393fe45ea44ac260da7d39fc8a1a87bf86
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269162
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Currently, to find users on the Microsoft side, we take their email,
username, or SIS id and look up the Microsoft user with the UPN
(UserPrincipalName) equal to that value. Thus, we call that
email/username/SIS id a "UPN" in the code.
Now however we are moving to a model with the user can choose what
target attribute on the Microsoft side they wish to match that value
(email/username/SIS id) to. (Additionally, they will be able to add an
arbitrary suffix to the value on the Canvas side before using it to look
up the user on the Microsoft side). Thus, we need a new term for this
value that we are calling a "UPN". I propose the term ULUV, or User
LookUp Value.
The only thing this commit does is rename constants/classes/variables.
refs INTEROP-6739
flag=microsoft_group_enrollments_syncing
Test plan:
- specs. as I add functionality in the next commits we will test that
the old functionality (looking up by UPN) still works.
Change-Id: I5898e9c0fd53c67c4f24dbf6166e67d68af8d9f3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269157
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ryan Hawkins <ryan.hawkins@instructure.com>
QA-Review: Ryan Hawkins <ryan.hawkins@instructure.com>
Product-Review: Evan Battaglia <ebattaglia@instructure.com>
Evan Battaglia