Bug 47374 - Container (ou) and sub objects are deleted on container rename with same name but different case
Container (ou) and sub objects are deleted on container rename with same name...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3-2-errata
Assigned To: Felix Botner
Arvid Requate
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-07-18 16:24 CEST by Jürn Brodersen
Modified: 2019-03-15 22:34 CET (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 4: A User would return the product
User Pain: 0.320
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jürn Brodersen univentionstaff 2018-07-18 16:24:31 CEST
Container (ou) and sub objects are deleted on container rename with same name but different case.

How to reproduce:
Add an ou
Add an user under that ou
Rename that ou using the same name but a different case
-> The ou is gone

See also test case:
63_udm-containers/07_container_ou_rename_uppercase
Comment 1 Felix Botner univentionstaff 2018-09-18 11:19:12 CEST
branch: fbotner/bug47374
test: http://jenkins.knut.univention.de:8080/job/UCS-4.3/job/UCS-4.3-2/view/Branch%20Tests/job/BranchTestBase/96/testReport/

When a ou is renamed in UCS, we 
 * create a temporary ou
 * move all objects from the old ou into the temporary ou
 * remove the old ou
 * create the new ou 
 * and move all objects from 

so during the rename of myOU to myou the connector sees the following steps

c ou=temp_ou
m uid=user1,ou=myOU -> uid=user1,ou=temp_ou
d ou=myOU
c ou=myou
m uid=user1,ou=temp_ou -> uid=user1,ou=myou

At some point the connector gets a delete "from_ucs" for ou=myOU and deletes that container in samba.

But later the connector checks deleted objects in samba and finds the old myOU object. This is a "to_ucs" delete for the old myOU but since our dn mapping and ldapsearch/ldapdelete (in general or in our ldap setup?) is case insensitive this "to_ucs" delete removes new ou myou.

Luckily, this is already half fixed. We store the objectGUID in "UCS deleted" after an object has been deleted in samba by the connector. So now we only have to check if the current objectGUID is already in "UCS deleted" during a delete to ucs. If so, we can ignore this delete as the ucs object has already been deleted.

TODO: merge
Comment 2 Felix Botner univentionstaff 2018-09-19 12:19:02 CEST
merged:

yaml: 9b3f7e0802cc4d858cf2f7acc43c22f16704d845
univention-s4-connector: f0066b2b5b35d90a59e7a08f0b1f183d7310e3e7 9e6de6b1335af564790ca66f5ee20e0bddc3ed18

added 52_s4connector/199sync_ucs_case_only_rename_ou
Comment 3 Felix Botner univentionstaff 2018-09-20 14:27:44 CEST
This change breaks 

52_s4connector.022_concurrent_deletion_in_ucs
52_s4connector.109sync_create_and_modify_ucs_to_ad

These tests create, modify and delete objects (users/groups) in UCS without waiting for the sync after every step. At the end the tests check if the objects is removed in UCS and fail if that is not the case.

In the log i see

delete_in_ucs: object uid=nxkgbntd,cn=users,dc=four,dc=three already deleted in UCS, ignoring delete

and this is the object the test checks. So with the change from this bug we no longer delete objects in UCS in certain cases.

Basically what happens is:

UCS 11:26:37  Object created: uid=nxkgbntd
UCS 11:26:40  Object modified: uid=nxkgbntd     

CON 11:26:38 sync from ucs: [user] [       add] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:26:38 sync from ucs: [user] [    modify] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:26:39 sync to ucs:   [user] [    modify] uid=nxkgbntd,cn=users,dc=four,dc=three
CON 11:26:45 sync from ucs: [user] [    modify] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:26:45 sync from ucs: [user] [    modify] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:26:46 sync to ucs:   [user] [    modify] uid=nxkgbntd,cn=users,dc=four,dc=three
CON 11:26:53 sync from ucs: [user] [    modify] cn=nxkgbntd,cn=users,DC=four,DC=three

UCS 11:27:00  Object removed: uid=nxkgbntd

CON 11:27:01 sync to ucs:   [user] [       add] uid=nxkgbntd,cn=users,dc=four,dc=three 
 -> modify converted to add by connector

CON 11:27:07 sync from ucs: [user] [    delete] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:27:07 sync from ucs: [user] [       add] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:27:07 sync from ucs: [user] [    modify] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:27:08 sync from ucs: [user] [    modify] cn=nxkgbntd,cn=users,DC=four,DC=three
CON 11:27:09 sync to ucs:   [user] [    modify] uid=nxkgbntd,cn=users,dc=four,dc=three
CON 11:27:09 sync to ucs:   [user] [    delete] uid=nxkgbntd,cn=users,dc=four,dc=three
CON 11:27:09 delete_in_ucs: object uid=nxkgbntd,cn=users,dc=four,dc=three already 
 deleted in UCS, ignoring delete
CON 11:27:15 sync from ucs: [user] [    modify] CN=nxkgbntd,CN=Users,DC=four,DC=three

UCS 11:27:17  test if uid=nxkgbntd exists, YES => fail

The connector gets lots of modify_from UCS which leads to modify_to_UCS.
At one point the UCS object is removed and the modify_to_UCS is converted to a add_to_UCS and the object is re-created in UCS.
Only now the connectors gets the delete_from_UCS and removes the object in samba.
Later the connectors finds its own (samba) delete and tries to remove the UCS object, which now fails with the added check.

Bottom line, the test wants the connector to delete objects (with the same DN) although it is a different object in terms of the objectGUID, the exact opposite of what this bug wants.

I am not sure how to handle this, except for applying this "delete test" only for ou's and container objects, feel free to re-open if there is a better solution.

yaml: 11a2e70bc24b04caa07539ca0b35b6143ec55531
univention-s4-connector: acaaa9a6aec8a97a264fa748a3b591cbec194d3d
Comment 4 Arvid Requate univentionstaff 2018-10-06 17:33:40 CEST
Quoting Comment 1:

> When a ou is renamed in UCS, we 
>  * create a temporary ou
>  * move all objects from the old ou into the temporary ou

Since the MDB backend supports subtree renames, we could change the behavior of python-univention-directory-manager to use that in case the slapd is running with that backend. Or we do a try/except style switch.

> This is a "to_ucs" delete for the old myOU but since our dn mapping and
> ldapsearch/ldapdelete (in general or in our ldap setup?) is case insensitive
> this "to_ucs" delete removes new ou myou.

By default all LDAP attributes have the syntax "DirectoryString" which apparently has a caseIgnoreSubstringsMatch by default (?). Anyway, it's LDAP as you can see with univention-ldapsearch ou="*Yo*".
Comment 5 Arvid Requate univentionstaff 2018-10-06 18:13:21 CEST
> univention-s4-connector: f0066b2b5b35d90a59e7a08f0b1f183d7310e3e7

Just a small thing: You do a self._dbcon.commit() but that's not required for a SELECT.

Code review: Ok
Test status in Jenkins: Ok
Advisory: Ok
Comment 6 Felix Botner univentionstaff 2018-10-08 10:14:12 CEST
(In reply to Arvid Requate from comment #5)
> > univention-s4-connector: f0066b2b5b35d90a59e7a08f0b1f183d7310e3e7
> 
> Just a small thing: You do a self._dbcon.commit() but that's not required
> for a SELECT.

yes, fixed

ad55ceb9de2d4ca11934a2613169256d895dc588 - univention-s4-connector
3defd60c6002f92854481f8a3c7284c878705348 - yaml
Comment 7 Arvid Requate univentionstaff 2018-10-08 16:08:29 CEST
Code-review: Ok
Package update and basic functional test: Ok
Advisory: Ok
Comment 8 Arvid Requate univentionstaff 2018-10-10 12:31:37 CEST
<http://errata.software-univention.de/ucs/4.3/266.html>