Bug 49324 - sync_to_ucs deleting GPO: delete of subobject failed cn=PushedPrinterConnections
sync_to_ucs deleting GPO: delete of subobject failed cn=PushedPrinterConnections
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 4.4-2-errata
Assigned To: Florian Best
Arvid Requate
:
: 45167 49950 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2019-04-18 14:38 CEST by Arvid Requate
Modified: 2019-11-12 06:35 CET (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.103
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support: Yes
Flags outvoted (downgraded) after PO Review:
Ticket number: 2019032721000316, 2019090421000581
Bug group (optional):
Max CVSS v3 score:


Attachments
idea.patch (1.61 KB, patch)
2019-04-18 15:23 CEST, Arvid Requate
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Comment 2 Arvid Requate univentionstaff 2019-04-18 15:23:30 CEST
Created attachment 9985 [details]
idea.patch

Probably same thing as Bug 41025 and Bug 45167. Untested patch attached.
Comment 3 Florian Best univentionstaff 2019-08-01 11:16:58 CEST
*** Bug 49950 has been marked as a duplicate of this bug. ***
Comment 4 Florian Best univentionstaff 2019-08-01 11:19:14 CEST
(In reply to Christina Scheinig from Bug #49950 comment #0)
> Maybe similar to Bug #49931 and/or Bug #49498 but different:
> =============================================================
> univention-s4connector-list-rejected
> 
> UCS rejected
> 
> 
> S4 rejected
> 
>     1:    S4 DN:
> cn={00B91A95-1EA6-42FF-BE15-9A7896448393},CN=Policies,CN=System,DC=schein,
> DC=me
>          UCS DN: <not found>
> 
>         last synced USN: 282008
>   01.08.2019 10:02:28.105 LDAP        (INFO   ): sync_to_ucs: set position
> to
> cn={00B91A95-1EA6-42FF-BE15-9A7896448393},cn=Policies,cn=System,dc=schein,
> dc=me
> 01.08.2019 10:02:28.106 LDAP        (INFO   ): LockingDB: Execute SQL
> command: 'SELECT id FROM UCS_LOCK WHERE uuid=?;',
> '('da633928-0551-1037-8386-5d830702ad99',)'
> 01.08.2019 10:02:28.107 LDAP        (INFO   ): LockingDB: Return SQL result:
> '[]'
> 01.08.2019 10:02:28.107 LDAP        (INFO   ): The following attributes have
> been changed: []
> 01.08.2019 10:02:28.108 LDAP        (INFO   ): sync_to_ucs: using existing
> target object type: container/cn
> 01.08.2019 10:02:28.425 LDAP        (INFO   ): delete object exception:
> Operation not allowed on non-leaf: subordinate objects must be deleted first
> 01.08.2019 10:02:28.426 LDAP        (INFO   ): remove object from UCS
> failed, need to delete subtree
> 01.08.2019 10:02:28.427 LDAP        (INFO   ): delete:
> cn=PushedPrinterConnections,cn=Machine,cn={00B91A95-1EA6-42FF-BE15-
> 9A7896448393},cn=Policies,cn=System,dc=schein,dc=me
> 01.08.2019 10:02:28.428 LDAP        (INFO   ): _object_mapping: map with key
> container and type ucs
> 01.08.2019 10:02:28.428 LDAP        (INFO   ): _dn_type ucs
> 01.08.2019 10:02:28.429 LDAP        (WARNING): delete subobject:
> cn=pushedprinterconnections,cn=machine,cn={00b91a95-1ea6-42ff-be15-
> 9a7896448393},cn=policies,cn=system,DC
> =schein,DC=me
> 01.08.2019 10:02:28.430 LDAP        (INFO   ): _ignore_object: Do not ignore
> cn=pushedprinterconnections,cn=machine,cn={00b91a95-1ea6-42ff-be15-
> 9a7896448393},cn=policies,
> cn=system,DC=schein,DC=me
> 01.08.2019 10:02:28.432 LDAP        (INFO   ): get_ucs_object: object found:
> cn=PushedPrinterConnections,cn=Machine,cn={00B91A95-1EA6-42FF-BE15-
> 9A7896448393},cn=Policies,
> cn=System,dc=schein,dc=me
> 01.08.2019 10:02:28.433 LDAP        (PROCESS): sync to ucs:   [    
> container] [    delete]
> cn=PushedPrinterConnections,cn=Machine,cn={00B91A95-1EA6-42FF-BE15-
> 9A789644839
> 3},cn=Policies,cn=System,dc=schein,dc=me
> 01.08.2019 10:02:28.433 LDAP        (INFO   ): sync_to_ucs: set position to
> cn=Machine,cn={00B91A95-1EA6-42FF-BE15-9A7896448393},cn=Policies,cn=System,
> dc=schein,dc=me
> 01.08.2019 10:02:28.434 LDAP        (INFO   ): LockingDB: Execute SQL
> command: 'SELECT id FROM UCS_LOCK WHERE uuid=?;',
> '('da85df0a-0551-1037-8388-5d830702ad99',)'
> 01.08.2019 10:02:28.435 LDAP        (INFO   ): LockingDB: Return SQL result:
> '[]'
> 01.08.2019 10:02:28.435 LDAP        (ERROR  ): Unknown Exception during
> sync_to_ucs
> 01.08.2019 10:02:28.436 LDAP        (ERROR  ): Traceback (most recent call
> last):
>   File
> "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line
> 1512, in sync_to_ucs
>     guid_unicode = original_object.get('attributes').get('objectGUID')[0]
> TypeError: 'NoneType' object has no attribute '__getitem__'·
> 
> 
> =============================================================
> I deleted the object in openLdap, because in
> /var/lib/samba/sysvol/domain/scripts the GPO was not there.
> eval "$(ucr shell)"
> ldapdelete -r -h "$ldap_master" -p 7389 -D "$ldap_hostdn" -y
> /etc/machine.secret
> "cn={00B91A95-1EA6-42FF-BE15-9A7896448393},cn=Policies,cn=System,dc=schein,
> dc=me"
> 
> Reject is solved
> 
> =============================================================
Comment 5 Christina Scheinig univentionstaff 2019-08-16 12:40:12 CEST
I set the ticket to waiting for Errata ant the bug to wating for support. The customer cannot update further Erratas without this Issue fixed.
Comment 6 Stefan Gohmann univentionstaff 2019-08-16 13:11:01 CEST
(In reply to Arvid Requate from comment #2)
> Created attachment 9985 [details]
> idea.patch
> 
> Probably same thing as Bug 41025 and Bug 45167. Untested patch attached.

AFAIR, con_subtree_delete_objects could be added to the s4 connector mapping. It could be configured by simply allowing containers to be deleted under containers (if that is the case here).
Comment 7 Florian Best univentionstaff 2019-09-24 14:55:10 CEST
(In reply to Arvid Requate from Bug #41025 comment #1)
> I guess we need to check which objectclass cn=microsoft and cn=microsoft
> have in cases like these. Maybe the con_subtree_delete_objects filter (Bug
> 33882) just needs to be extended. If that is the case, then Bug 45311 should
> be considered too.
Comment 8 Florian Best univentionstaff 2019-09-24 19:43:37 CEST
The complexity of the code here is very high.
Here is a stacktrace of where the exception happens: 

  /usr/lib/python2.7/dist-packages/univention/s4connector/s4/main.py(300)<module>()
-> main()
  /usr/lib/python2.7/dist-packages/univention/s4connector/s4/main.py(283)main()
-> connect()
  /usr/lib/python2.7/dist-packages/univention/s4connector/s4/main.py(196)connect()
-> s4.initialize()
  /usr/lib/python2.7/dist-packages/univention/s4connector/s4/__init__.py(2075)initialize()
-> self.resync_rejected()
  /usr/lib/python2.7/dist-packages/univention/s4connector/s4/__init__.py(2108)resync_rejected()
-> sync_successfull = self.sync_to_ucs(property_key, mapped_object, dn, object)
  /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1548)sync_to_ucs()
-> result = self.delete_in_ucs(property_type, object, module, position)
  /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1427)delete_in_ucs()
-> if not self.sync_to_ucs(key, subobject_ucs, back_mapped_subobject['dn'], object):
  /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1548)sync_to_ucs()
-> result = self.delete_in_ucs(property_type, object, module, position)
  /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1427)delete_in_ucs()
-> if not self.sync_to_ucs(key, subobject_ucs, back_mapped_subobject['dn'], object):
> /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1505)sync_to_ucs()
-> guid = str(ndr_unpack(misc.GUID, guid_blob))

sync_to_ucs()
  calls delete_in_ucs()
    calls sync_to_ucs()
      calls delete_in_ucs()
        calls sync_to_ucs()
          fails with the exception from comment #0

Analysing this:

1. The original synchronization to UCS:
> sync_to_ucs(property_key, mapped_object, dn, object)
(Pdb) property_key
'msGPO'
(Pdb) mapped_object
{'dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},CN=Policies,CN=System,dc=four,dc=four', 'attributes': {'distinguishedName': [u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four'], 'cn': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8'], 'objectClass': [u'top', u'container', u'groupPolicyContainer'], 'isRecycled': [u'TRUE'], 'objectGUID': [u'3[)\x86g\x92\xadI\xab\xd4\x03\n\x97\xde\xc0\xd8'], 'whenChanged': [u'20190924161417.0Z'], 'lastKnownParent': [u'CN=Policies,CN=System,DC=four,DC=four'], 'whenCreated': [u'20190924161357.0Z'], 'uSNCreated': [u'4076'], 'uSNChanged': [u'4083'], 'instanceType': [u'4'], 'isDeleted': [u'TRUE'], 'name': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8']}, 'deleted_dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four', 'modtype': 'delete', 'changed_attributes': []}
(Pdb) dn
u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},CN=Policies,CN=System,DC=four,DC=four'
(Pdb) object
{'dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},CN=Policies,CN=System,DC=four,DC=four', 'attributes': {'distinguishedName': [u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four'], 'cn': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8'], 'objectClass': [u'top', u'container', u'groupPolicyContainer'], 'isRecycled': [u'TRUE'], 'objectGUID': [u'3[)\x86g\x92\xadI\xab\xd4\x03\n\x97\xde\xc0\xd8'], 'whenChanged': [u'20190924161417.0Z'], 'lastKnownParent': [u'CN=Policies,CN=System,DC=four,DC=four'], 'whenCreated': [u'20190924161357.0Z'], 'uSNCreated': [u'4076'], 'uSNChanged': [u'4083'], 'instanceType': [u'4'], 'isDeleted': [u'TRUE'], 'name': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8']}, 'deleted_dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four', 'modtype': 'delete'}

2. causes the removal of the msgpo policy object in UCS
>   calls delete_in_ucs(property_type, object, module, position)
(Pdb) property_type
'msGPO'
(Pdb) object
{'dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},CN=Policies,CN=System,dc=four,dc=four', 'attributes': {'distinguishedName': [u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four'], 'cn': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8'], 'objectClass': [u'top', u'container', u'groupPolicyContainer'], 'isRecycled': [u'TRUE'], 'objectGUID': [u'3[)\x86g\x92\xadI\xab\xd4\x03\n\x97\xde\xc0\xd8'], 'whenChanged': [u'20190924161417.0Z'], 'lastKnownParent': [u'CN=Policies,CN=System,DC=four,DC=four'], 'whenCreated': [u'20190924161357.0Z'], 'uSNCreated': [u'4076'], 'uSNChanged': [u'4083'], 'instanceType': [u'4'], 'isDeleted': [u'TRUE'], 'name': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8']}, 'deleted_dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four', 'modtype': 'delete', 'changed_attributes': []}
(Pdb) module
<module 'univention.admin.handlers.container.msgpo' from '/usr/lib/pymodules/python2.7/univention/admin/handlers/container/msgpo.pyc'>
(Pdb) position.getDn()
'CN=Policies,CN=System,dc=four,dc=four'

→ This fails because the msgpo has subobjects.
ucs_object.remove() raises:
(Pdb) e
ldapError('Operation not allowed on non-leaf: subordinate objects must be deleted first',)
(Pdb) e.original_exception
NOT_ALLOWED_ON_NONLEAF({'info': 'subordinate objects must be deleted first', 'desc': 'Operation not allowed on non-leaf'},)


3. The exception handling now iterates over all subobjects and tries to remove them (by calling a fake sync_to_ucs with modtype=delete ...):
>     calls sync_to_ucs(key, subobject_ucs, back_mapped_subobject['dn'], object)
> /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1427)delete_in_ucs()
-> if not self.sync_to_ucs(key, subobject_ucs, back_mapped_subobject['dn'], object):
(Pdb) key
'container'
(Pdb) subobject_ucs
{'dn': 'cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four', 'attributes': {'hasSubordinates': ['TRUE'], 'entryCSN': ['20190924161358.759701Z#000000#000#000000'], 'cn': ['Machine'], 'objectClass': ['top', 'organizationalRole', 'univentionObject'], 'univentionObjectType': ['container/cn'], 'entryUUID': ['0f2bd43a-7332-1039-811c-ddbff1682aec'], 'modifyTimestamp': ['20190924161358Z'], 'modifiersName': ['cn=admin,dc=four,dc=four'], 'createTimestamp': ['20190924161358Z'], 'entryDN': ['cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four'], 'subschemaSubentry': ['cn=Subschema'], 'structuralObjectClass': ['organizationalRole'], 'creatorsName': ['cn=admin,dc=four,dc=four']}, 'changed_attributes': [], 'modtype': 'delete'}
(Pdb) back_mapped_subobject['dn']
'cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,DC=four,DC=four'
(Pdb) object
{'dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},CN=Policies,CN=System,dc=four,dc=four', 'attributes': {'distinguishedName': [u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four'], 'cn': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8'], 'objectClass': [u'top', u'container', u'groupPolicyContainer'], 'isRecycled': [u'TRUE'], 'objectGUID': [u'3[)\x86g\x92\xadI\xab\xd4\x03\n\x97\xde\xc0\xd8'], 'whenChanged': [u'20190924161417.0Z'], 'lastKnownParent': [u'CN=Policies,CN=System,DC=four,DC=four'], 'whenCreated': [u'20190924161357.0Z'], 'uSNCreated': [u'4076'], 'uSNChanged': [u'4083'], 'instanceType': [u'4'], 'isDeleted': [u'TRUE'], 'name': [u'{9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\nDEL:86295b33-9267-49ad-abd4-030a97dec0d8']}, 'deleted_dn': u'CN={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A}\\0ADEL:86295b33-9267-49ad-abd4-030a97dec0d8,CN=Deleted Objects,DC=four,DC=four', 'modtype': 'delete', 'changed_attributes': []}
(Pdb) object['attributes'].keys()
['distinguishedName', 'cn', 'objectClass', 'isRecycled', 'objectGUID', 'whenChanged', 'lastKnownParent', 'whenCreated', 'uSNCreated', 'uSNChanged', 'instanceType', 'isDeleted', 'name']

→ Here we still have a valid definition which can be used to remove the object.

4. Now it tries to remove the sub-object e.g. cn=Machine,$policy. 
>       calls delete_in_ucs()
(Pdb) down
> /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1548)sync_to_ucs()
-> result = self.delete_in_ucs(property_type, object, module, position)
(Pdb) property_type
'container'
(Pdb) object
{'dn': 'cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four', 'attributes': {'hasSubordinates': ['TRUE'], 'entryCSN': ['20190924161358.759701Z#000000#000#000000'], 'cn': ['Machine'], 'objectClass': ['top', 'organizationalRole', 'univentionObject'], 'univentionObjectType': ['container/cn'], 'entryUUID': ['0f2bd43a-7332-1039-811c-ddbff1682aec'], 'modifyTimestamp': ['20190924161358Z'], 'modifiersName': ['cn=admin,dc=four,dc=four'], 'createTimestamp': ['20190924161358Z'], 'entryDN': ['cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four'], 'subschemaSubentry': ['cn=Subschema'], 'structuralObjectClass': ['organizationalRole'], 'creatorsName': ['cn=admin,dc=four,dc=four']}, 'changed_attributes': [], 'modtype': 'delete'}
(Pdb) module
<module 'univention.admin.handlers.container.cn' from '/usr/lib/pymodules/python2.7/univention/admin/handlers/container/cn.pyc'>
(Pdb) position.getDn()
'cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four'

→ That removal fails again because that object also has subobjects.

5. The exception handling wants to remove also those subobjects.
>         calls sync_to_ucs(property_key, mapped_object, dn, object)
> /usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py(1427)delete_in_ucs()
-> if not self.sync_to_ucs(key, subobject_ucs, back_mapped_subobject['dn'], object):
(Pdb) key
'container'
(Pdb) subobject_ucs
{'dn': 'cn=PushedPrinterConnections,cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four', 'attributes': {'hasSubordinates': ['TRUE'], 'entryCSN': ['20190924161358.788629Z#000000#000#000000'], 'cn': ['PushedPrinterConnections'], 'objectClass': ['top', 'organizationalRole', 'univentionObject'], 'univentionObjectType': ['container/cn'], 'entryUUID': ['0f303e3a-7332-1039-811e-ddbff1682aec'], 'modifyTimestamp': ['20190924161358Z'], 'modifiersName': ['cn=admin,dc=four,dc=four'], 'createTimestamp': ['20190924161358Z'], 'entryDN': ['cn=PushedPrinterConnections,cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four'], 'subschemaSubentry': ['cn=Subschema'], 'structuralObjectClass': ['organizationalRole'], 'creatorsName': ['cn=admin,dc=four,dc=four']}, 'modtype': 'delete'}
(Pdb) back_mapped_subobject['dn']
u'cn=pushedprinterconnections,cn=machine,cn={9ad0eada-b0ff-dead-beef-ce040e4e136a},cn=policies,cn=system,DC=four,DC=four'
(Pdb) object
{'dn': 'cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four', 'attributes': {'hasSubordinates': ['TRUE'], 'entryCSN': ['20190924161358.759701Z#000000#000#000000'], 'cn': ['Machine'], 'objectClass': ['top', 'organizationalRole', 'univentionObject'], 'univentionObjectType': ['container/cn'], 'entryUUID': ['0f2bd43a-7332-1039-811c-ddbff1682aec'], 'modifyTimestamp': ['20190924161358Z'], 'modifiersName': ['cn=admin,dc=four,dc=four'], 'createTimestamp': ['20190924161358Z'], 'entryDN': ['cn=Machine,cn={9AD0EADA-B0FF-DEAD-BEEF-CE040E4E136A},cn=Policies,cn=System,dc=four,dc=four'], 'subschemaSubentry': ['cn=Subschema'], 'structuralObjectClass': ['organizationalRole'], 'creatorsName': ['cn=admin,dc=four,dc=four']}, 'changed_attributes': [], 'modtype': 'delete'}
(Pdb) e
ldapError('Operation not allowed on non-leaf: subordinate objects must be deleted first',)
(Pdb) e.original_exception
NOT_ALLOWED_ON_NONLEAF({'info': 'subordinate objects must be deleted first', 'desc': 'Operation not allowed on non-leaf'},)
(Pdb) object['attributes'].keys()
['hasSubordinates', 'entryCSN', 'cn', 'objectClass', 'univentionObjectType', 'entryUUID', 'modifyTimestamp', 'modifiersName', 'createTimestamp', 'entryDN', 'subschemaSubentry', 'structuralObjectClass', 'creatorsName']

This is the place where Arvid's patch from comment #2 starts.
We see, that we have a `object` which is not a s4-object but a ucs-object and therefore has no `objectGUID` set.
I think Arvids idea was to replace this wrong dictionary with a correct one containing a S4-object-definition.
This approach unfortionately doesn't work because it causes that this logic is already applied in frame number 3.
In frame 3 we have such a valid object definition. With the patch it would search with scope=base in the S4-LDAP to an object which is already removed.
This would fail with a NO_SUCH_OBJECT exception.

So, I think the real bug is:
exception handling in delete_in_ucs() must not call sync_to_ucs() with the original `object` but needs to give something mapped or the original object as well.
Otherwise recursion causes these exception:

>           fails with the exception from comment #0

Tomorrow I will try to find the real fix for it.
Comment 9 Florian Best univentionstaff 2019-09-25 20:14:00 CEST
The real reason for the broken removal is the following:

The mapping from the deleted DN to the original DN does not work if the parent object is also already removed.
__dn_from_deleted_object() contained already a FIXME note. I implemented the necessary recursion.

The relevant log lines are:
24.09.2019 18:02:08.308 LDAP        (PROCESS): sync to ucs:   [msPrintConnectionPolicy] [    delete] CN={7C18D2B5-7A83-4FB8-B28E-965D9B54C518},CN=PushedPrinterConnections
DEL:0576a22f-1aa3-497d-97e2-bd1f98e448d7,CN=Deleted Objects,dc=four,dc=four
24.09.2019 18:02:08.309 LDAP        (WARNING): Object to delete doesn't exists, ignore (CN={7C18D2B5-7A83-4FB8-B28E-965D9B54C518},CN=PushedPrinterConnections
DEL:0576a22f-1aa3-497d-97e2-bd1f98e448d7,CN=Deleted Objects,dc=four,dc=four)
24.09.2019 18:02:08.316 LDAP        (PROCESS): sync to ucs:   [     container] [    delete] CN=PushedPrinterConnections,CN=Machine
DEL:9e070627-9748-49de-9bce-bf40d9b0fae1,CN=Deleted Objects,dc=four,dc=four
24.09.2019 18:02:08.316 LDAP        (WARNING): Object to delete doesn't exists, ignore (CN=PushedPrinterConnections,CN=Machine
DEL:9e070627-9748-49de-9bce-bf40d9b0fae1,CN=Deleted Objects,dc=four,dc=four)
24.09.2019 18:02:08.323 LDAP        (PROCESS): sync to ucs:   [     container] [    delete] CN=Machine,CN={9AD0EADA-B0FF-DEAD-BEEF-628B35BBC539}
DEL:166500c8-9bd9-4443-b382-c8c6e60792c0,CN=Deleted Objects,dc=four,dc=four
24.09.2019 18:02:08.323 LDAP        (WARNING): Object to delete doesn't exists, ignore (CN=Machine,CN={9AD0EADA-B0FF-DEAD-BEEF-628B35BBC539}
DEL:166500c8-9bd9-4443-b382-c8c6e60792c0,CN=Deleted Objects,dc=four,dc=four)
24.09.2019 18:02:08.329 LDAP        (PROCESS): sync to ucs:   [     container] [    delete] CN=User,CN={9AD0EADA-B0FF-DEAD-BEEF-628B35BBC539}
DEL:166500c8-9bd9-4443-b382-c8c6e60792c0,CN=Deleted Objects,dc=four,dc=four
24.09.2019 18:02:08.330 LDAP        (WARNING): Object to delete doesn't exists, ignore (CN=User,CN={9AD0EADA-B0FF-DEAD-BEEF-628B35BBC539}
DEL:166500c8-9bd9-4443-b382-c8c6e60792c0,CN=Deleted Objects,dc=four,dc=four)

There you can see that the DN's is "CN={7C18D2B5-7A83-4FB8-B28E-965D9B54C518},CN=PushedPrinterConnections\nDEL:0576a22f-1aa3-497d-97e2-bd1f98e448d7,CN=Deleted Objects,dc=four,dc=four".
The S4-Connector uses the lastKnownParent attribute of deleted objects to build the original DN of the to be removed object. That of course failed if the lastKnownParent contains also a DN of a removed object (with "\nDEL: ...").
Comment 10 Florian Best univentionstaff 2019-09-25 20:26:08 CEST
(In reply to Stefan Gohmann from comment #6)
> AFAIR, con_subtree_delete_objects could be added to the s4 connector
> mapping. It could be configured by simply allowing containers to be deleted
> under containers (if that is the case here).
(In reply to Arvid Requate from Bug #41025 comment #1)
> I guess we need to check which objectclass cn=microsoft and cn=microsoft
> have in cases like these. Maybe the con_subtree_delete_objects filter (Bug
> 33882) just needs to be extended. If that is the case, then Bug 45311 should
> be considered too.

FYI: Using "con_subtree_delete_objects" would not work here!:
con_subtree_delete_objects is for removing objects in Samba4-LDAP (delete_in_s4()).

The bug occurrs in the opposite case, during removal of objects in UCS-LDAP (delete_in_ucs()).

There is no `ucs_subtree_delete_objects`, which could be used in delete_in_ucs() currently. And I think it's not necesarry to add one currently.

As this bug is fixed by fixing __dn_from_deleted_object(), it is not really necessary to have working code which removes subtree-objects.
Comment 11 Florian Best univentionstaff 2019-09-25 20:55:55 CEST
Patch available in git:fbest/49324-subobjects-removal (especially git:45b6d5f0433dcc9166899a0eea81fc4505ab714c).
Branch Test: https://jenkins.knut.univention.de:8181/job/UCS%20Branch%20Test/184/
Will merge tomorrow.
Comment 12 Florian Best univentionstaff 2019-09-27 10:37:24 CEST
A test cases which reproduces the issue has been added.
The removal of objects where the parent object doesn't exists has been repaired.
I did not fix the broken subtree-removal handling, should I fix it here, as well?
Otherwise I will clone the bug.

ucs-test (9.0.3-60)
cf306b7559a3 | Bug #49324: Merge branch 'fbest/49324-subobjects-removal' into 4.4-2
82b97c50343f | Bug #49324: debian/changelog

univention-s4-connector (13.0.2-47)
953fa677d285 | Bug #49324: fix typo

univention-s4-connector (13.0.2-46)
cf306b7559a3 | Bug #49324: Merge branch 'fbest/49324-subobjects-removal' into 4.4-2
82b97c50343f | Bug #49324: debian/changelog

ucs-test (9.0.3-59)
9a3f4b15f537 | Bug #49324: add test case 52_s4connector/503_gpo_removal_with_parents_already_removed

univention-s4-connector.yaml
2e1526a3875e | YAML Bug #49324

univention-s4-connector (13.0.2-45)
74f70fd6d70e | Bug #49324: give arguments to univention-s4-connector
13073160c187 | Bug #49324: enhance ldap.NOT_ALLOWED_ON_NONLEAF error handling
b80cca1f713e | Bug #49324: enhance readability of looping over mapping
b51c18aca3eb | Bug #49324: remove code duplication
50360ce8614a | Bug #49324: fix removal of objects where the parent object is already removed
b12706997f51 | Bug #49324: rename method into __identify_s4_type
20a6461cea44 | Bug #49324: log tracebacks of ignored exceptions
4512fcbbe3fd | Bug #49324: fix exception handling
70f0218f342c | Bug #49324: rename variable into samba_object and change_usn
117e9c2806d1 | Bug #49324: add various docstrings for better understanding
6406616b5766 | Bug #49324: remove impossible exception handling
cd4bc5e7398c | Bug #49324: rename variable into samba_object
52a58ba50e8c | Bug #49324: pep8
Comment 13 Arvid Requate univentionstaff 2019-10-02 15:08:15 CEST
Verified:
* Code review
* New test case works
* Advisory
Comment 14 Erik Damrose univentionstaff 2019-10-02 15:54:58 CEST
<http://errata.software-univention.de/ucs/4.4/298.html>
Comment 15 Florian Best univentionstaff 2019-11-12 06:35:23 CET
*** Bug 45167 has been marked as a duplicate of this bug. ***