Bug 37685 - deletion of school fails
deletion of school fails
Status: REOPENED
Product: UCS@school
Classification: Unclassified
Component: UMC - Wizards
UCS@school 4.4
Other Linux
: P5 normal with 4 votes (vote)
: ---
Assigned To: UCS@school maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-02-04 13:52 CET by Florian Best
Modified: 2020-11-11 09:38 CET (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.069
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:
best: Patch_Available+


Attachments
failed_school.ldif (14.42 KB, text/plain)
2015-02-04 13:56 CET, Florian Best
Details
patch (git:fbest/37685-removal-of-school) (3.24 KB, patch)
2019-10-14 14:10 CEST, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2015-02-04 13:52:03 CET
Traceback:
Die Ausführung des Kommandos schoolwizards/schools/remove schoolwizards/schools ist fehlgeschlagen:

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/__init__.py",
line 176, in _decorated
    return function(self, request, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 204, in
wrapper_func
    return func( *args, **kwargs )
  File
"/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py",
line 118, in _decorated
    ret = func(self, request, *a, **kw)
  File
"/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py",
line 233, in _delete_obj
    if obj.remove(ldap_user_write):
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 501, in remove
    success = self.remove_without_hooks(lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 511, in
remove_without_hooks
    udm_obj.remove(remove_childs=True)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 525, in
remove
    return self._remove(remove_childs)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1034, in
_remove
    self.lo.delete(self.dn)
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 464, in delete
    raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg)
ldapError: Operation not allowed on non-leaf: subordinate objects must be deleted first

After having this incomplete school I cannot use the users wizards anymore:

Die Ausführung des Kommandos schoolwizards/classes schoolwizards/users ist fehlgeschlagen:

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/__init__.py", line 176, in _decorated
    return function(self, request, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 204, in wrapper_func
    return func( *args, **kwargs )
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 589, in classes
    self.finished( request.id, self._groups( ldap_user_read, search_base.school, search_base.classes, request.options.get('pattern') ) )
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 581, in _groups
    groupresult = udm_modules.lookup('groups/group', None, ldap_connection, scope = scope, base = ldap_base, filter = ldapFilter)
  File "/usr/lib/pymodules/python2.7/univention/admin/modules.py", line 801, in lookup
    tmpres=module.lookup(co, lo, filter, base=base, superordinate=superordinate, scope=scope, unique=unique, required=required, timeout=timeout, sizelimit=sizelimit)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/groups/group.py", line 1086, in lookup
    for dn, attrs in lo.search(unicode(filter), base, scope, [], unique, required, timeout, sizelimit):
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 345, in search
    raise univention.admin.uexceptions.noObject(_err2str(msg))
noObject: No such object: cn=groups,ou=8058,dc=mydomain,dc=intranet
Comment 1 Florian Best univentionstaff 2015-02-04 13:56:44 CET
Created attachment 6642 [details]
failed_school.ldif

A LDIF of everything underneath of the OU what's there after the removal. Removing again results in the same error.
Comment 2 Florian Best univentionstaff 2015-02-04 14:03:10 CET
The cause is that there still exist some exam users.
Comment 4 Sönke Schwardt-Krummrich univentionstaff 2019-02-05 21:20:38 CET
This issue has been filled against UCS@school 4.0. The maintenance with bug 
and security fixes for UCS@school 4.0 has ended on May 31, 2016.

Customers still on UCS 4.0 are encouraged to update to UCS 4.3 (or later). 
Please contact your partner or Univention for any questions.

If this issue still occurs in newer UCS versions, please use "Clone this bug"
or simply reopen the issue. In this case please provide detailed information on
how this issue is affecting you.
Comment 5 Florian Best univentionstaff 2019-09-27 11:40:36 CEST
http://jenkins.knut.univention.de:8080/job/UCSschool-4.4/view/Ergebnisse/job/Install%20Multiserver%20Large%20Environment/195/default/testReport/master300.90_ucsschool/106_valid_hostname/master300_master300/

(2019-09-26 08:54:27.045528) 2019-09-26 08:54:27 INFO  ucs_test_school.cleanup:276  UCSTestSchool cleanup done
(2019-09-26 08:54:27.048564) Traceback (most recent call last):
(2019-09-26 08:54:27.048595)   File "106_valid_hostname", line 124, in <module>
(2019-09-26 08:54:27.048646)     main()
(2019-09-26 08:54:27.048671)   File "106_valid_hostname", line 89, in main
(2019-09-26 08:54:27.048693)     process_school_umcp(school, dc_name, should_fail=False)
(2019-09-26 08:54:27.048716)   File "106_valid_hostname", line 71, in process_school_umcp
(2019-09-26 08:54:27.048736)     school.remove()
(2019-09-26 08:54:27.048759)   File "/usr/lib/pymodules/python2.7/univention/testing/ucsschool/school.py", line 184, in remove
(2019-09-26 08:54:27.048805)     reqResult = self.client.umc_command('schoolwizards/schools/remove', param, flavor).result
(2019-09-26 08:54:27.048821)   File "/usr/lib/python2.7/dist-packages/univention/testing/umc.py", line 59, in umc_command
(2019-09-26 08:54:27.048832)     return super(Client, self).umc_command(*args, **kwargs)
(2019-09-26 08:54:27.048844)   File "/usr/lib/python2.7/dist-packages/univention/lib/umc.py", line 435, in umc_command
(2019-09-26 08:54:27.049497)     return self.request('POST', 'command/%s' % (path,), data, headers)
(2019-09-26 08:54:27.049526)   File "/usr/lib/python2.7/dist-packages/univention/testing/umc.py", line 70, in request
(2019-09-26 08:54:27.049572)     response = super(Client, self).request(method, path, data, headers)
(2019-09-26 08:54:27.049597)   File "/usr/lib/python2.7/dist-packages/univention/lib/umc.py", line 515, in request
(2019-09-26 08:54:27.049640)     return self.send(request)
(2019-09-26 08:54:27.049663)   File "/usr/lib/python2.7/dist-packages/univention/lib/umc.py", line 544, in send
(2019-09-26 08:54:27.049709)     raise HTTPError(request, response, self.hostname)
(2019-09-26 08:54:27.050002) univention.lib.umc.HTTPError: 591 on master300.autotest300.local (command/schoolwizards/schools/remove):
{'location': 'https://master300.autotest300.local/univention/command',
 'message': 'Interner Server-Fehler in "schoolwizards/schools/remove (schoolwizards/schools)".',
 'status': 591,
 'traceback': """
Interner Server-Fehler in "schoolwizards/schools/remove (schoolwizards/schools)".
Request: schoolwizards/schools/remove (schoolwizards/schools)

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/management/console/base.py", line 260, in execute
    function.__func__(self, request, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 181, in _response
    return function(self, request)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py", line 122, in _decorated
    ret = func(self, request, *a, **kw)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/schoolldap.py", line 145, in wrapper_func
    return func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/schoolwizards/__init__.py", line 257, in _delete_obj
    if obj.remove(ldap_user_write):
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 657, in remove
    success = self.remove_without_hooks(lo)
  File "/usr/lib/pymodules/python2.7/ucsschool/lib/models/base.py", line 667, in remove_without_hooks
    udm_obj.remove(remove_childs=True)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 877, in remove
    return self._remove(remove_childs)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 1525, in _remove
    self.lo.delete(self.dn)
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 970, in delete
    raise univention.admin.uexceptions.ldapError(_err2str(msg), original_exception=msg)
ldapError: Operation not allowed on non-leaf: subordinate objects must be deleted first
"""
}
Comment 6 Florian Best univentionstaff 2019-09-27 15:22:42 CEST
If this happens again in Jenkins we need: /var/log/univention/management-console-module-schoolwizards.log
Comment 7 Florian Best univentionstaff 2019-09-27 15:25:44 CEST
Ideas why this can happen:
* a not identifyable object exists somewhere underneath of the OU (e.g. a uninstalled app with UDM handlers)
* the S4-Connector re-recreates objects during the removal of the school(? just a theory)
Comment 8 Florian Best univentionstaff 2019-10-02 16:17:55 CEST
Another idea:
If there is a msGPO policy object (container/msgpo) or settings/msprintconnectionpolicy or settings/mswmifilter underneath of the school and the schools gets removed on a DC Slave where no python-univention-connector-s4 is installed, the removal will also fail, because the UDM modules aren't installed there.
Comment 9 Florian Best univentionstaff 2019-10-14 14:02:01 CEST
The reason is the following:
11.10.19 11:14:22.908  MODULE      ( PROCESS ) : Deleting School(name='oldschool', dn='ou=oldschool,l=school,l=dev')
11.10.19 11:14:22.909  MODULE      ( PROCESS ) : Deleting School(name='oldschool', dn='ou=oldschool,l=school,l=dev')
11.10.19 11:14:23.203  ADMIN       ( ERROR   ) : remove: could not remove 'cn=Domain Users oldschool,cn=groups,ou=oldschool,l=school,l=dev': primaryGroupUsed:
11.10.19 11:14:23.205  ADMIN       ( ERROR   ) : remove: could not remove 'cn=groups,ou=oldschool,l=school,l=dev': ldapError: Operation not allowed on non-leaf: subordinate objects must be deleted first
11.10.19 11:14:23.456  MODULE      ( PROCESS ) : Interner Server-Fehler in "schoolwizards/schools/remove (schoolwizards/schools)"

The group "Domain Users $school" is found first in the tree of subobjects to be removed before the users which are part of the group are removed.

This can reliable be reproduced when a exam user exists, the order of removal seems:
* remove uid=*,cn=users,
* remove cn=*,cn=groups (which fails)
* remove uid=*,cn=exam-users,
Comment 10 Florian Best univentionstaff 2019-10-14 14:10:51 CEST
Created attachment 10203 [details]
patch (git:fbest/37685-removal-of-school)
Comment 11 Sönke Schwardt-Krummrich univentionstaff 2019-10-18 10:52:57 CEST
(In reply to Florian Best from comment #10)
> Created attachment 10203 [details]
> patch (git:fbest/37685-removal-of-school)

Hmmm... if I'm right this patch currently causes an error message in the first attempt of removing the childs of the OU, where the group is first tried to delete, and then everything is fine after the second run.

Is it possible, for example, that first the users are deleted and then the groups, without adding a lot of magic to the remove() of containers/ou?
Comment 12 Florian Best univentionstaff 2019-10-18 11:15:04 CEST
(In reply to Sönke Schwardt-Krummrich from comment #11)
> (In reply to Florian Best from comment #10)
> > Created attachment 10203 [details]
> > patch (git:fbest/37685-removal-of-school)
> 
> Hmmm... if I'm right this patch currently causes an error message in the
> first attempt of removing the childs of the OU, where the group is first
> tried to delete, and then everything is fine after the second run.
Yes
 
> Is it possible, for example, that first the users are deleted and then the
> groups, without adding a lot of magic to the remove() of containers/ou?

We could make a [x.remove() for x in module.get('users/user').lookup(base=self.dn, scope=subtree)] before the subtree removal block.
But there might be similar cases with computer-references? Not sure.
Comment 13 Daniel Tröder univentionstaff 2019-10-22 08:39:35 CEST
I suggest:

------------------------------------------------------------
udm = UDM.admin().version(0)

dns = sort_by_length(searchDN(base=$OU), reversed=True)

for dn in dns:
    try:
        udm.obj_by_dn(dn).delete()
    except UnknownModuleType:
        warn("... $dn")
        lo.delete(dn)
------------------------------------------------------------

The "sort_by_length(.., reversed=True)" will ensure to remove leafs before branches.
Comment 14 Florian Best univentionstaff 2019-10-22 09:27:05 CEST
(In reply to Daniel Tröder from comment #13)
> The "sort_by_length(.., reversed=True)" will ensure to remove leafs before
> branches.
This is not the problem of the bug. The problem is that first groups are removed and then users which is the wrong order because you cannot remove groups which are the primary group of a user.
Comment 15 Florian Best univentionstaff 2020-11-11 09:38:09 CET
*** Bug 52346 has been marked as a duplicate of this bug. ***