Bug 50460 - [O365] app update to multiconnection data model
[O365] app update to multiconnection data model
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Office 365
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: Arvid Requate
Erik Damrose
:
Depends on:
Blocks: 50579
  Show dependency treegraph
 
Reported: 2019-11-06 17:47 CET by Erik Damrose
Modified: 2020-02-13 09:49 CET (History)
3 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Erik Damrose univentionstaff 2019-11-06 17:47:13 CET
Followup to Bug #50433

Migrate single ad connection to multiple ad connection data model, implementation has to be done in the migration script itself and the office365 listener modules
Comment 1 Arvid Requate univentionstaff 2019-11-19 11:40:39 CET
Commits for this Bug:

01724ec | Fix command order in joinscript
f436fd4 | Fix command order in joinscript
ef5a911 | Add univentionOffice365ADConnections syntax and
          Office365ADConnectionsHook
d867468 | Add translation file
5441936 | Add translation file
db9a728 | Deactivate migration script call in joinscript for now
37ea715 | Adjust migration script
2f4f2aa | Use Office365Listener.en/decode_o365data everywhere
f65abd9 | Call manage_connections after creating the containers in joinscript
5c51ee0 | only sync groups with users in matching AAD
819a87d | After automatic migration remove the legacy extended attributes
Comment 2 Erik Damrose univentionstaff 2019-11-19 17:48:14 CET
As discussed, assigning an ADconnection in UMC and trying to save the user object produces the following error:

Interner Server-Fehler in "udm/put (users/user)".

Request: udm/put (users/user)

  File "/usr/lib/pymodules/python2.7/notifier/threads.py", line 78, in _run
    tmp = self._function()
  File "/usr/lib/pymodules/python2.7/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/__init__.py", line 440, in _thread
    module.modify(properties)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/udm_ldap.py", line 645, in modify
    obj.modify()
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/users/user.py", line 1433, in modify
    return super(object, self).modify(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 651, in modify
    dn = self._modify(modify_childs, ignore_license=ignore_license, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1323, in _modify
    ml = self.call_udm_property_hook('hook_ldap_modlist', self, ml)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1057, in call_udm_property_hook
    changes = func(module, changes)
  File "/usr/lib/pymodules/python2.7/univention/admin/hooks.d/office365_user_ADConnections_hook.py", line 73, in hook_ldap_modlist
    new_adconnection_data[adconnection] = self.adconnection_data[adconnection]
AttributeError: 'Office365ADConnectionsHook' object has no attribute 'adconnection_data'
Comment 3 Arvid Requate univentionstaff 2019-11-19 21:06:17 CET
Fixed:

5bef4a0 | Update univentionOffice365ADConnectionAlias in hook
bf7ad8b | Update univentionOffice365ADConnectionAlias in hook
Comment 4 Erik Damrose univentionstaff 2019-11-20 16:24:57 CET
Reopen: When trying to activate a user for o365 and assigning a domainalias, i get the following error when saving the user in UMC:

Die folgenden Eigenschaften konnten nicht validiert werden:
Azure AD connection,User Principal Name:
super(type, obj): obj must be an instance or subtype of type

univention-office365 2.0.2-34A~4.4.0.201911201138 on a new system
Comment 5 Florian Best univentionstaff 2019-11-20 22:19:51 CET
Suggested patch (untested):

diff --git modules/univention/syntax.d/office365.py modules/univention/syntax.d/office365.py
index 84f237c..0fb014a 100644
--- modules/univention/syntax.d/office365.py
+++ modules/univention/syntax.d/office365.py
@@ -71,7 +71,7 @@ class univentionOffice365ADConnections(complex):
        all_required = True

        @classmethod
-       def parse(self, texts):
-               p = super(univentionOffice365ADConnections, self).parse(texts)
+       def parse(cls, texts):
+               p = super(cls, cls).parse(texts)
                objectID, userPrincipalName = p
                return p
Comment 6 Arvid Requate univentionstaff 2019-11-20 23:00:11 CET
Yes, thanks, that fixed the issue!

5c222f1 | Fix inheritance issue in univentionOffice365ADConnections syntax parser
Comment 7 Erik Damrose univentionstaff 2019-11-25 17:59:55 CET
OK: UDM syntax works
OK: user migration generally works

reopen: I think we could improve the migration process:
* The migration script fetches and prints the user ucsschoolSchool attribute for no apparent reason
* The o365Data attribute is a bit messy after the migration script has run: The script itself sets the correct values. But the new listener is already running, which fetches the old.o365Data object and uses dict.update() to create the new o365Data values, which in effect overwrites the value that was just set in the migration script. Now the attribute contains the old value plus the new adconnection. The only key in the o365Data JSON should be the 'defaultADconnection', but it also contains the old value, the azure user object representation, as well.

>>> univention.office365.listener.Office365Listener.decode_o365data("eJylVFuP0kAU/iubPgu7BZFlnySUNSTAYqga1vgw7ZzSA9OZZma6a9f43z1TCi2oUeNjp+f6Xc43L2fGPCvNV1olKMC7u5KFEK+uPFNoyTL34BUSn0BaVNL36E9hQK8ts63gjKFYYrz/TQb1wK0EPscYpAFDIZ+/0DuHhBXCjoNYSQmxi6df36oOK40yxpyJ5c8136okoVL9N4Muh6dO84c+XT8V7ajajLu0wShhI38YdW5HN286r5Nk2IlYBB0+hP7I70W3/eHQ+05JghkboF6XMg4xay3339PYFPSCEKr2PsvMSq4IO9lFaTWTYD0HC6FhKIIoeUIO/F7pBUqlzycKy7waZAFZBNr1gSwXqgSo9q4jFWeWde0xFmOtjEpslxYlhJQu16CfaHjT/WAOVQrDtjBXMavZ8IJpA+mx6TE4Rls2zY40rwSTJ47/mQvKSVhsMCM5hiAgT5WEZVFteWoVqyxnsqzZOEr2XJQnnU5SJrfAH2Q7VAPYMecajGmet46a86K5orJioviF3FsRGhLQmuRNbQpCr4VIHKtC2qlkkQAHgdUF1KPNuFOBxcYOVIfGSUO1J5d8ZAL5vVZZQBucCzLWULFTs1G/7lQUom1bGM2EYKISaKClCcyywrqBDqRswvflQzB+2ezGdhMud8v1zddFsBjMw/3z4tPMbl7uxTJ43C2C6evHYNP3KuPmTNuMFmjDoL6WNaLNTpmKzs4KR5MLVv7CRq6uwXzVKtOkCdgyMd7CO62KfCKc0shzR43WQfZ3ajkdOSUwPgB+9IdcaXD4mACNReIPTQr8QgJpaaiZCEDQtLp8qPx+EePMamicC/lzmqniihgkIWX5BY3AW/x6vRt/1PH9Tm8Q+v7dwL/r9R4rnzkV6ZbVbEorSpLhKlVWvT3YPAOObMrRzlHuXTl+dPlDZUJz/TcGvP7Tlbg+b+61tycEp1orfdrfXYRZZakGksORbUzxExdriAtNp+VgkQTbXJpUPc9kLZA5kdZyW62P+uX7D4PaVVk=")

{u'passwordPolicies': None, u'passwordProfile': None, u'surname': u'univention1', u'userState': None, u'mailNickname': u'univention1', u'assignedLicenses': [], u'defaultADconnection': {u'userPrincipalName': u'univention1@office365.dev-univention.de', u'objectId': u'59fa917b-8906-4ff7-babe-d7e3912b8377'}, u'lastDirSyncTime': None, u'userPrincipalName': u'univention1@office365.dev-univention.de', u'sipProxyAddress': None, u'consentProvidedForMinor': None, u'userType': u'Member', u'usageLocation': u'DE', u'objectType': u'User', u'city': None, u'assignedPlans': [], u'objectId': u'59fa917b-8906-4ff7-babe-d7e3912b8377', u'signInNames': [], u'facsimileTelephoneNumber': None, u'creationType': None, u'streetAddress': None, u'userStateChangedOn': None, u'state': None, u'otherMails': [u'univention1@mydomain.intranet'], u'mail': None, u'legalAgeGroupClassification': None, u'accountEnabled': True, u'userIdentities': [], u'refreshTokensValidFromDateTime': None, u'companyName': None, u'jobTitle': None, u'isCompromised': None, u'immutableId': u'YTQyODAzYjAtYTNjNS0xMDM5LTkwMWItYzFlNDZjMDE4ZDY3', u'postalCode': None, u'proxyAddresses': [], u'department': None, u'physicalDeliveryOfficeName': None, u'employeeId': None, u'telephoneNumber': None, u'odata.type': u'Microsoft.DirectoryServices.User', u'onPremisesDistinguishedName': None, u'displayName': u'univention1', u'provisionedPlans': [], u'deletionTimestamp': None, u'mobile': None, u'country': None, u'thumbnailPhoto@odata.mediaEditLink': u'directoryObjects/59fa917b-8906-4ff7-babe-d7e3912b8377/Microsoft.DirectoryServices.User/thumbnailPhoto', u'provisioningErrors': [], u'createdDateTime': u'2019-11-25T11:51:22Z', u'preferredLanguage': None, u'dirSyncEnabled': None, u'showInAddressList': None, u'onPremisesSecurityIdentifier': None, u'givenName': None, u'ageGroup': None}
Comment 8 Arvid Requate univentionstaff 2019-11-26 22:06:43 CET
4829e87 | Disgard legacy content of UniventionOffice365Data after migration
4e63313 | Don't print ucsschoolSchool
178988e | Set default connection if user just got activated
d174e57 | Changelog
cd6cffa | Remove office365_userPrincipalName_hook.py
c5c58bf | Remove office365_userPrincipalName_hook.py
4574d38 | Migrate config files to new directory regardless of the value of UCRV office365/migrate/adconnectionalias.
bd5bba3 | Create defaultADConnection during update and migrate existing json files to subdir
c6f877c | Don't initialize defaultADConnection directory during (re-)join
1793311 | Fix adconnection_id in ids.json on package update
7be46ed | Some code cleanup
Comment 9 Erik Damrose univentionstaff 2019-11-27 17:49:12 CET
3bd709c6 Remove obsolete check from migration script

OK: App update migrates all users, unless UCRv is set
OK: UDM extended attributes are updated and shown according to the migration state
Comment 10 Erik Damrose univentionstaff 2019-11-28 14:26:21 CET
OK: with ucr set office365/migrate/adconnectionalias=false before the migration, no user gets migrated to the new data format. New users can still be created and are synced to the azure ad.
OK~ no modifications to user attributes (e.g. firstname) is synced to azure until the modification script ahs run again
OK: after the migration script is called, all users are migrated and their attributes reflect the UCS state.

Verified
Comment 11 Erik Damrose univentionstaff 2020-02-13 09:49:44 CET
Closed: Released with App Version 3.0 for UCS 4.4