Bug 53551 - udm/put (users/user): TypeError: unhashable type: 'list' with office365 on a backup
udm/put (users/user): TypeError: unhashable type: 'list' with office365 on a ...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Office 365
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: Julia Bremer
Dirk Wiesenthal
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2021-07-07 09:14 CEST by Christina Scheinig
Modified: 2021-09-13 17:23 CEST (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.069
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2021070221000441, 2021082021000271
Bug group (optional):
Max CVSS v3 score:
best: Patch_Available+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christina Scheinig univentionstaff 2021-07-07 09:14:44 CEST
A customer gets this traceback, when trying to set AD Connection via UMC.
Office365 connector is installed on a Backup server. The UMC of the Master is used for the adjustment.

-----------------------------
Interner Server-Fehler in "udm/put (users/user)".
Request: udm/put (users/user)

   File "/usr/lib/python2.7/dist-packages/notifier/threads.py", line 80, in _run
     result = self._function()
   File "/usr/lib/python2.7/dist-packages/notifier/__init__.py", line 104, in __call__
     return self._function(*tmp, **self._kwargs)
   File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/__init__.py", line 440, in _thread 
     module.modify(properties)
   File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/udm_ldap.py", line 645, in modify
     obj.modify() 
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/users/user.py", line 1410, in modify
     return super(object, self).modify(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 650, in modify
     dn = self._modify(modify_childs, ignore_license=ignore_license, response=response)
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1326, in _modify
     ml = self._ldap_object_classes(ml) 
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1439, in _ldap_object_classes
     newattr = ldap.cidict.cidict(_MergedAttributes(self, ml).get_attributes())
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 3612, in get_attributes
     return dict((attr, self.get_attribute(attr)) for attr in attributes)
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 3612, in <genexpr>
     return dict((attr, self.get_attribute(attr)) for attr in attributes)
   File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 3623, in get_attribute
     values |= set(new)
TypeError: unhashable type: 'list'
--------------------

Setting the attribute via udm it works fine:
udm users/user modify
--dn="uid=sun.devschueler,cn=schueler,cn=users,ou=SUN,dc=schein,dc=schule" --set
UniventionOffice365ADConnectionAlias=sunADconnection
udm users/user modify
--dn="uid=sun.devschueler,cn=schueler,cn=users,ou=SUN,dc=schein,dc=schule" --set
UniventionOffice365Enabled=1

--------------------
Also adjusting the UniventionOffice365ADConnectionAlias on the UMC of the Backup, does not show the traceback.

--------------------
univention-app info:

Backup:
-------------
Installed: office365=3.4 prometheus-node-exporter=1.1 ucsschool=4.4 v9

Master:
-------------
UCS: 4.4-8 errata974
Installed: prometheus-node-exporter=1.1 ucsschool=4.4 v9
ucsschool-veyon-proxy=1.1
Comment 1 Florian Best univentionstaff 2021-07-13 14:27:37 CEST
Internal server error during "udm/put (users/user)".
Request: udm/put (users/user)

  File "/usr/lib/python2.7/dist-packages/notifier/threads.py", line 80, in _run
    result = self._function()
  File "/usr/lib/python2.7/dist-packages/notifier/__init__.py", line 104, in __call__
    return self._function(*tmp, **self._kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/__init__.py", line 440, in _thread
    module.modify(properties)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/udm/udm_ldap.py", line 645, in modify
    obj.modify()
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/users/user.py", line 1410, in modify
    return super(object, self).modify(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 650, in modify
    dn = self._modify(modify_childs, ignore_license=ignore_license, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/handlers/__init__.py", line 1330, in _modify
    self.dn = self.lo.modify(self.dn, ml, ignore_license=ignore_license, serverctrls=serverctrls, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/admin/uldap.py", line 891, in modify
    return self.lo.modify(dn, changes, serverctrls=serverctrls, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 207, in _decorated
    return func(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 756, in modify
    self.modify_ext_s(dn, ml, serverctrls=serverctrls, response=response)
  File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 207, in _decorated
    return func(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 813, in modify_ext_s
    rtype, rdata, rmsgid, resp_ctrls = self.lo.modify_ext_s(dn, ml, serverctrls=serverctrls)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 987, in modify_ext_s
    return self._apply_method_s(SimpleLDAPObject.modify_ext_s,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 931, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 373, in modify_ext_s
    msgid = self.modify_ext(dn,modlist,serverctrls,clientctrls)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 370, in modify_ext
    return self._ldap_call(self._l.modify_ext,dn,modlist,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls))
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call
    result = func(*args,**kwargs)
TypeError: ('expected a string in the list', ['defaultADconnection', ''])
Comment 2 Florian Best univentionstaff 2021-07-13 14:35:44 CEST
(In reply to Florian Best from comment #1)
corresponding modlist is:
[(0, 'dummyUniventionOffice365ADConnections', [['defaultADconnection', '']])]
Comment 3 Florian Best univentionstaff 2021-07-13 15:08:22 CEST
DC Backup:

# univention-app info
UCS: 4.4-8 errata974
Installed: office365=3.4 prometheus-node-exporter=1.1 ucsschool=4.4 v9
Upgradable: 

# dpkg -l univention-office365 univention-management-console-module-office365
================================================================
ii  univention-management-console-module-office365  2.0.2-100A~4.4.0.20210115150 all                          Office 365 setup wizard UMC module
ii  univention-office365                            2.0.2-100A~4.4.0.20210115150 all                          Provision MS Office 365 accounts.


DC Master:
# univention-app info
UCS: 4.4-8 errata974
Installed: prometheus-node-exporter=1.1 self-service=4.0 self-service-backend=4.0 ucsschool=4.4 v9 ucsschool-veyon-proxy=1.1
Upgradable: 


# dpkg -l univention-office365 univention-management-console-module-office365
dpkg-query: Kein Paket gefunden, das auf univention-office365 passt
dpkg-query: Kein Paket gefunden, das auf univention-management-console-module-office365 passt


To further enhance this I registered the hook on the DC Backup so it's available on the DC Master:
# ucs_registerLDAPExtension --packageversion 2.0.2-100A~4.4.0.20210115150 --packagename univention-office365 --udm_hook /usr/share/pyshared/univention/admin/hooks.d/office365_user_ADConnections_hook.py
Object exists: cn=udm_hook,cn=univention,dc=base
Object created: cn=office365_user_ADConnections_hook,cn=udm_hook,cn=univention,dc=base

Waiting for activation of the extension object office365_user_ADConnections_hook: .OK
Waiting for file /usr/share/pyshared/univention/admin/hooks.d/office365_user_ADConnections_hook.py: OK
Comment 4 Florian Best univentionstaff 2021-07-13 15:10:15 CEST
Now we get the traceback from comment 0, where att, old, new corresponds to:

('dummyUniventionOffice365ADConnections', [], [[u'defaultADconnection', u'']])
Comment 5 Florian Best univentionstaff 2021-07-13 15:39:14 CEST
The extended attribute "cn=UniventionOffice365ADConnections,cn=custom attributes,cn=univention" is the only Office 356 extended attribute which defines hook=Office365ADConnectionsHook.

Reproducer:
import univention.debug as ud
ud.init('stdout', ud.FLUSH, ud.NO_FUNCTION)
ud.set_level(ud.ADMIN, 4)
import univention.admin.uldap
lo,po = univention.admin.uldap.getMachineConnection()
univention.admin.modules.update()
u = univention.admin.modules.get('users/user')
univention.admin.modules.init(lo, po, u)

s = u.object(None, lo, po, 'uid=devschueler,cn=schueler,cn=users,ou=XX,dc=base')
s.open()
s["UniventionOffice365Enabled"] = True
s["UniventionOffice365ADConnections"] = [["defaultADconnection",""]]
s.modify()

→ The hook is not loaded.

(Pdb) [x.hook for x in m.extended_udm_attributes if x.name == 'UniventionOffice365ADConnections']
[None]

Logfile / says:

ADMIN       ( ERROR   ) : admin.hook.import_hook_files: loading '/usr/lib/python2.7/dist-packages/univention/admin/hooks.d/office365_user_ADConnections_hook.py' failed
ADMIN       ( ERROR   ) : admin.hook.import_hook_files: TRACEBACK:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/admin/hook.py", line 56, in import_hook_files
    exec(fd, sys.modules[__name__].__dict__)
  File "/usr/lib/python2.7/dist-packages/univention/admin/hooks.d/office365_user_ADConnections_hook.py", line 35, in <module>
    from univention.office365.listener import Office365Listener
ImportError: No module named office365.listener

→ The hook has a hard dependency on univention-office356. Therefore the package needs to be added to the DefaultPackagesMaster in the app.ini file.
Comment 6 Florian Best univentionstaff 2021-07-13 15:52:01 CEST
I fixed it in the customer environment with the fix in:

https://git.knut.univention.de/univention/components/office365/-/commits/fbest/53551-hotfix
Comment 7 Julia Bremer univentionstaff 2021-08-27 18:43:19 CEST
I applied the patch by florian:
9edcfec Bug #53551: fix univention-office356 only installed on a DC Backup
Comment 8 Dirk Wiesenthal univentionstaff 2021-09-10 10:18:04 CEST
Original error: Fixed
Joinscript: OK
Comment 9 Erik Damrose univentionstaff 2021-09-13 17:23:02 CEST
Published in App 'office365' version 4.0