Bug 50349 - schoolwizards/computers/query: KeyError: 'ucsschoolRole'
schoolwizards/computers/query: KeyError: 'ucsschoolRole'
Status: CLOSED FIXED
Product: UCS@school
Classification: Unclassified
Component: UMC - Computer room administration
UCS@school 4.4
Other Linux
: P5 normal (vote)
: UCS@school 4.4 v4-errata
Assigned To: Ole Schwiegert
Jürn Brodersen
:
Depends on: 49311
Blocks:
  Show dependency treegraph
 
Reported: 2019-10-11 12:45 CEST by Christina Scheinig
Modified: 2019-11-27 17:09 CET (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.114
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Ticket number: 2019092421000785, 2019101821000267, 2019110521001298, 2019112121000464
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christina Scheinig univentionstaff 2019-10-11 12:45:07 CEST
A customer gets this Traceback, if he opens computers (schools) just like Bug 49311

Interner Server-Fehler in "schoolwizards/computers/query (schoolwizards/computers)".
Request: schoolwizards/computers/query (schoolwizards/computers)

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/management/console/base.py", line 260, in execute
    function.__func__(self, request, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 181, in _response
    return function(self, request)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py", line 122, in _decorated
    ret = func(self, request, *a, **kw)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 145, in wrapper_func
    return func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py", line 330, in get_computers
    return self._get_all(computer_class, school, request.options.get('filter'), ldap_user_read)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py", line 271, in _get_all
    objs.extend(klass.get_all(lo, school.name, filter_str=filter_str, easy_filter=True))
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 818, in get_all
    ret.append(cls.from_udm_obj(udm_obj, school, lo))
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 323, in from_udm_obj
    obj = super(SchoolComputer, cls).from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 869, in from_udm_obj
    return klass.from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 323, in from_udm_obj
    obj = super(SchoolComputer, cls).from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 876, in from_udm_obj
    udm_value = udm_obj[attr.udm_name]
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 477, in __getitem__
    elif key not in self.__no_default and self.descriptions[key].editable:
KeyError: 'ucsschoolRole'

The customer already has the Bugfix installed on his server:
------------------------------------
python-ucs-school:
  Installiert:           12.1.7A~4.4.0.201909041215
  Installationskandidat: 12.1.7A~4.4.0.201909041215
  Versionstabelle:
 *** 12.1.7A~4.4.0.201909041215 500
        500 https://appcenter.software-univention.de/univention-repository/4.4/maintained/component ucsschool_20190723074557/all/ Packages
        100 /var/lib/dpkg/status
     12.1.5A~4.4.0.201908271804 500
        500 https://appcenter.software-univention.de/univention-repository/4.4/maintained/component ucsschool_20190723074557/all/ Packages
     12.1.4A~4.4.0.201907301859 500
        500 https://appcenter.software-univention.de/univention-repository/4.4/maintained/component ucsschool_20190723074557/all/ Packages
     12.1.2-1A~4.4.0.201907090931 500
        500 https://appcenter.software-univention.de/univention-repository/4.4/maintained/component ucsschool_20190723074557/all/ Packages
     11.0.2-6A~4.3.0.201904091600 500
        500 https://appcenter.software-univention.de/univention-repository/4.3/maintained/component ucsschool_20190723074557/all/ Packages
     11.0.2-5A~4.3.0.201902251246 500
        500 https://appcenter.software-univention.de/univention-repository/4.3/maintained/component ucsschool_20190723074557/all/ Packages

---------------------------------------
So I am not sure this is a regression. I tried to fix the school roles but got a traceback.


/usr/share/ucs-school-import/scripts/fix_ucsschool_roles --dry-run

------------------------

Changing "ucsschoolRole" of 'cn=paedagogik,cn=dc,cn=server,cn=computers,ou=sun,dc=schein,dc=me' from ['dc_slave_edu:school:sun', 'win                                   _computer:school:sun'] to ['dc_slave_edu:school:sun', 'dc_slave_edu:school:sun'].
Traceback (most recent call last):
  File "/usr/share/ucs-school-import/scripts/fix_ucsschool_roles", line 92, in <module>
    lo.modify(dn, [('ucsschoolRole', old_roles, new_roles)])
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 902, in modify
    raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg)
univention.admin.uexceptions.ldapError: Type or value exists: ucsschoolRole: value #0 provided more than once

-----------------------------------------
The traceback could be replaced with a readable error message? 

This script fixed the problem.
/usr/share/ucs-school-import/scripts/fix_ucsschool_slaves --dry-run

I am not sure why this situation could occure in the first place? Maybe tis is not a bug at all, but debugging the situation was quite difficult.
Comment 1 Florian Best univentionstaff 2019-10-18 08:49:02 CEST
Reported again, Version: 4.4-2 errata298 (Blumenthal) - UCS@school 4.4 v2

Remark: auf Master über Schuladministration auf Kategorie Rechner gecklickt
Comment 2 Christian Völker univentionstaff 2019-11-08 15:18:15 CET
Interner Server-Fehler in "computerroom/room/acquire".
Request: computerroom/room/acquire

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 358, in __error_handling
    six.reraise(etype, exc, etraceback)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 261, in execute
    function.__func__(self, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 181, in _response
    return function(self, request)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 145, in wrapper_func
    return func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/computerroom/__init__.py", line 392, in room_acquire
    self._italc.room = roomDN
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/computerroom/italc2.py", line 647, in room
    self._set(value)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 145, in wrapper_func
    return func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/computerroom/italc2.py", line 709, in _set
    computers = list(computerroom.get_computers(lo))
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/group.py", line 288, in get_computers
    yield SchoolComputer.from_dn(host, self.school, ldap_connection)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 935, in from_dn
    return cls.from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 323, in from_udm_obj
    obj = super(SchoolComputer, cls).from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 869, in from_udm_obj
    return klass.from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 323, in from_udm_obj
    obj = super(SchoolComputer, cls).from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 876, in from_udm_obj
    udm_value = udm_obj[attr.udm_name]
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 478, in __getitem__
    elif key not in self.__no_default and self.descriptions[key].editable:
KeyError: 'ucsschoolRole'
Comment 3 Christian Völker univentionstaff 2019-11-08 15:18:59 CET
b) Modul Schuladministration/Computerräume verwalten:

Interner Server-Fehler in "schoolrooms/computers".
Request: schoolrooms/computers

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 358, in __error_handling
    six.reraise(etype, exc, etraceback)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 261, in execute
    function.__func__(self, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 181, in _response
    return function(self, request)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 145, in wrapper_func
    return func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolrooms/__init__.py", line 66, in computers
    } for x in SchoolComputer.get_all(ldap_user_read, request.options['school'], pattern)]
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 818, in get_all
    ret.append(cls.from_udm_obj(udm_obj, school, lo))
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 323, in from_udm_obj
    obj = super(SchoolComputer, cls).from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 869, in from_udm_obj
    return klass.from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/computer.py", line 323, in from_udm_obj
    obj = super(SchoolComputer, cls).from_udm_obj(udm_obj, school, lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 876, in from_udm_obj
    udm_value = udm_obj[attr.udm_name]
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 478, in __getitem__
    elif key not in self.__no_default and self.descriptions[key].editable:
KeyError: 'ucsschoolRole'
Comment 5 Daniel Tröder univentionstaff 2019-11-12 16:56:27 CET
There is a problem with the OpenLDAP ACLs.
The machine account can read the attribute "ucsschoolRole" on user and group objects, but a "normal" user cannot. (Rooms are technically groups.)
Comment 6 Ole Schwiegert univentionstaff 2019-11-13 15:51:13 CET
The tracebacks show a problem with the (School)Computer objects and not the SchoolRoom(groups) objects though
Comment 7 Ole Schwiegert univentionstaff 2019-11-14 09:42:12 CET
I checked a bit on one customers system and identified several computer objects in computer rooms that, for some reasons, do not have the objectClass ucsschoolComputer and thus do not have an ucsschoolRole. We still have to determine why this situation could occur
Comment 8 Ole Schwiegert univentionstaff 2019-11-14 12:56:45 CET
Okay we think that we found the reason of the entire problem. With the fix of Bug #49311 we changed the SchoolComputer.lookup function in a way that it only returns computer objects containing the objectClass ucsschoolComputer.

The migration script migrate_ucsschool_roles retrieves all SchoolComputers of any given school by using SchoolComputer.get_all which in turn uses the lookup function.

That means that after the mentioned fix the migration script stopped working properly since not all objects that needed modification were collected anymore.

Thus we have to modify the migration script now to use an alternative way to fetch the objects.
Comment 9 Ole Schwiegert univentionstaff 2019-11-14 13:30:03 CET
Proposed fix is in oschwieg/4.4/50349

When all objects or computers-school is specified, all macos, windows and ipmanaged objects under the OU are fetched and the appropriate school roles are attached.
Comment 10 Christina Scheinig univentionstaff 2019-11-25 10:37:17 CET
A customer, who already had removed the windows role from his school slaves, now ran in the same problem and the win_computer role is set again on the slaves.

Where does this come from, that the roles are wrongly assigned again?

------------------------
Looking for affected domaincontroller_slave objects...
Will modify: cn=slave1,cn=dc,cn=server,cn=computers,ou=sun,dc=school,dc=schein,dc=me
Roles: {'new': ['dc_slave_edu:school:sun'], 'old': ['dc_slave_edu:school:sun', 'win_computer:school:sun']}
ObjectClass: {'new': ['krb5KDCEntry', 'person', 'top', 'univentionHost', 'univentionDomainController', 'univentionObject', 'univentionNagiosHostClass', 'krb5Principal', 'shadowAccount', 'posixAccount', 'sambaSamAccount', 'univentionPolicyReference', 'univentionPortalComputer', 'ucsschoolServer'], 'old': ['krb5KDCEntry', 'person', 'ucsschoolComputer', 'top', 'univentionHost', 'univentionDomainController', 'univentionObject', 'univentionWindows', 'univentionNagiosHostClass', 'krb5Principal', 'shadowAccount', 'posixAccount', 'sambaSamAccount', 'univentionPolicyReference', 'univentionPortalComputer', 'ucsschoolServer']}
DRY-RUN: skipping modification

-------------------------

root@master:~# apt-cache policy python-ucs-school
python-ucs-school:
  Installiert:           12.1.9A~4.4.0.201911031011
  Installationskandidat: 12.1.9A~4.4.0.201911031011
Comment 11 Ole Schwiegert univentionstaff 2019-11-25 11:41:51 CET
(In reply to Christina Scheinig from comment #10)
> A customer, who already had removed the windows role from his school slaves,
> now ran in the same problem and the win_computer role is set again on the
> slaves.
> 
> Where does this come from, that the roles are wrongly assigned again?
> 
> ------------------------
> Looking for affected domaincontroller_slave objects...
> Will modify:
> cn=slave1,cn=dc,cn=server,cn=computers,ou=sun,dc=school,dc=schein,dc=me
> Roles: {'new': ['dc_slave_edu:school:sun'], 'old':
> ['dc_slave_edu:school:sun', 'win_computer:school:sun']}
> ObjectClass: {'new': ['krb5KDCEntry', 'person', 'top', 'univentionHost',
> 'univentionDomainController', 'univentionObject',
> 'univentionNagiosHostClass', 'krb5Principal', 'shadowAccount',
> 'posixAccount', 'sambaSamAccount', 'univentionPolicyReference',
> 'univentionPortalComputer', 'ucsschoolServer'], 'old': ['krb5KDCEntry',
> 'person', 'ucsschoolComputer', 'top', 'univentionHost',
> 'univentionDomainController', 'univentionObject', 'univentionWindows',
> 'univentionNagiosHostClass', 'krb5Principal', 'shadowAccount',
> 'posixAccount', 'sambaSamAccount', 'univentionPolicyReference',
> 'univentionPortalComputer', 'ucsschoolServer']}
> DRY-RUN: skipping modification
> 
> -------------------------
> 
> root@master:~# apt-cache policy python-ucs-school
> python-ucs-school:
>   Installiert:           12.1.9A~4.4.0.201911031011
>   Installationskandidat: 12.1.9A~4.4.0.201911031011

This problem should be connected to some issues Sönke identified in the S4-Connector. He documented that in Bug #50280. Please put the infos/questions there since in this bug we want to fix the regression from the migration script.
Comment 12 Jürn Brodersen univentionstaff 2019-11-26 18:31:03 CET
Branchtest looks good:
https://jenkins.knut.univention.de:8181/job/UCS%20Branch%20Test/240/testReport/90_ucsschool/

Removed objectClass and role -> Script works again

I would suggest to narrow down the ldap search a little:
[oschwieg/4.4/50349 0a425ee23] Bug #50349: Narrow down the ldapsearch

But your version works just as well :)

-> OK for merge onto 4.4
Comment 13 Jürn Brodersen univentionstaff 2019-11-26 18:48:08 CET
SDB Article:
https://help.univention.com/t/keyerror-ucsschoolrole-in-computerroom-room-aquire-and-schoolwizards-computers-query/13718

I think the article should include the traceback (which I now fail to reproduce?). The article states that the migration script is called again on updated. Does the join script version need to be adjusted or did I miss something?
Comment 14 Ole Schwiegert univentionstaff 2019-11-27 10:54:10 CET
Package: ucs-school-import
Version: 17.0.18A~4.4.0.201911271049
Branch: ucs_4.4-0
Scope: ucs-school-4.4

The migration script is *not run* automatically during the update
A note was added to the changelog for 4.4v5
The SDB article now includes the tracebacks and explains how to run the script again.
Comment 15 Jürn Brodersen univentionstaff 2019-11-27 13:35:42 CET
Small wording changes:
[4.4 fc22d40fd] Bug #50349: changelog/yaml wording
I also changed the date format in the sdb article

What I tested:
win/mac/ip without ucsschoolrole -> OK
win/mac/ip without objectClass and ucsschoolrole -> OK
sdb article -> OK
changelog/yaml -> OK

-> Verified
Comment 16 Jürn Brodersen univentionstaff 2019-11-27 14:32:34 CET
Missed this one:
[4.4 32abda11d] Bug #50349: fix debian/changelog syntax

I don't think a rebuild is necessary.
Comment 17 Jürn Brodersen univentionstaff 2019-11-27 17:09:05 CET
UCS@school 4.4 v4 has been released (errata update to the release).

https://docs.software-univention.de/changelog-ucsschool-4.4v4-de.html

If this error occurs again, please clone this bug.