Bug 41711 - Fix UDM locking/unlocking
Fix UDM locking/unlocking
Status: REOPENED
Product: UCS
Classification: Unclassified
Component: UDM (Generic)
UCS 4.4
Other Linux
: P5 normal with 9 votes (vote)
: ---
Assigned To: UMC maintainers
:
: 7337 12125 14798 18339 20631 23338 23868 26221 28632 29684 31414 31416 42150 42444 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-06-30 16:33 CEST by Florian Best
Modified: 2019-11-13 10:31 CET (History)
15 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.103
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Ticket number: 2016081821000428, 2017010321002241, 2017042421000191, 2017112121000315, 2018113021000243
Bug group (optional): Cleanup, Error handling, External feedback
Max CVSS v3 score:
best: Patch_Available+


Attachments
patch (needs rebase with svn r74405) (93.32 KB, patch)
2016-06-30 16:59 CEST, Florian Best
Details | Diff
patch (git:fbest/41711-fix-udm-locking) (58.07 KB, patch)
2019-05-09 20:06 CEST, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2016-06-30 16:33:21 CEST
UDM locking and unlocking of specific attributes follow the same structure everywhere. But the implementation is broken in very much situations.

The structure is:
Ask to lock attribute when creating, or raise AlreadyUsed if lock exists.
Confirm lock after successful creation. Revert(Release) locks if creation failed.

Ask to lock attribute when modifying (the attribute), or raise AlreadyUsed if lock exists.
Confirm lock of the previous old value if the modification was successful.
Confirm lock after successful modification. Revert(Release) locks if modification failed.

Release (value) lock when removing a object was successful.

This must work when using the same UDM object for all operations and if all operations are done with a new object. E.g:
o = object()
o[uid] = 1; o.create();
o[uid] = 2; o.modify()
o.delete()
o[uid] = 1; o.create()

A generic implementation should handle this.
My idea is to add an flag 'requires_lock' to the property()-description.
Comment 1 Florian Best univentionstaff 2016-06-30 16:59:49 CEST
Created attachment 7779 [details]
patch
(needs rebase with svn r74405)

The patch makes the above things generic.
TODO: implement the idea with the generic property flag
TODO: also cleanup a little bit more the releasing of changed values in _ldap_modlist

The patch fixes Bug #41294 and Bug #31414.
Comment 2 Florian Best univentionstaff 2016-06-30 17:00:04 CEST
*** Bug 31414 has been marked as a duplicate of this bug. ***
Comment 3 Florian Best univentionstaff 2016-06-30 17:00:12 CEST
*** Bug 41294 has been marked as a duplicate of this bug. ***
Comment 4 Florian Best univentionstaff 2016-06-30 17:04:44 CEST
*** Bug 23338 has been marked as a duplicate of this bug. ***
Comment 5 Florian Best univentionstaff 2016-07-01 14:47:38 CEST
*** Bug 14798 has been marked as a duplicate of this bug. ***
Comment 6 Jan Christoph Ebersbach univentionstaff 2016-08-18 16:27:55 CEST
At Ticket #2016081821000428 a partner reported a similar problem as described at Bug #23338.
Comment 7 Alexander Kläser univentionstaff 2016-08-23 16:31:37 CEST
I observed a similar issue during one of my talks:

-------- 8< ----------
root@ucs:~# last_id=$(cat /var/lib/univention-ldap/last_id)
root@ucs:~# udm computers/linux create \
    --position "cn=computers,$ldap_base" \
    --set name=mylinux \
    --set mac=52:54:00:9b:e5:01 \
    --set network="cn=default,cn=networks,$ldap_base"

Object created: cn=mylinux,cn=computers,dc=univention-gmbh,dc=intranet

root@ucs:~# sed -n "/^$((last_id+1))/,\$ p" \
    /var/lib/univention-ldap/notify/transaction
956 cn=default,cn=networks,dc=univention-gmbh,dc=int... m
957 cn=10.200.26.3,cn=aRecord,cn=temporary,cn=univen... a
958 cn=2006,cn=uidNumber,cn=temporary,cn=univention,... a
959 cn=2006,cn=gidNumber,cn=temporary,cn=univention,... a
960 cn=2006,cn=gidNumber,cn=temporary,cn=univention,... d
961 cn=S-1-5-21-2047279768-839456739-1946195436-5012,\
    cn=sid,cn=temporary,cn=univention,dc=univention-... a
962 cn=52:54:00:9b:e5:01,cn=mac,cn=temporary,cn=univ... a
963 cn=uidNumber,cn=temporary,cn=univention,dc=unive... m
964 cn=2006,cn=uidNumber,cn=temporary,cn=univention,... d
965 cn=mylinux$,cn=uid,cn=temporary,cn=univention,d.... a
966 cn=mylinux,cn=computers,dc=univention-gmbh,dc=in... a
967 cn=mylinux$,cn=uid,cn=temporary,cn=univention,dc... d
968 cn=mylinux,cn=computers,dc=univention-gmbh,dc=in... m
969 cn=mylinux,cn=computers,dc=univention-gmbh,dc=in... m
970 cn=Computers,cn=groups,dc=univention-gmbh,dc=int... m
971 relativeDomainName=mylinux,zoneName=univention-gmbh.\
    intranet,cn=dns,dc=univention-gmbh,dc=intranet m
972 relativeDomainName=3,zoneName=26.200.10.in-addr.arpa,\
    cn=dns,dc=univention-gmbh,dc=intranet a
973 zoneName=26.200.10.in-addr.arpa,cn=dns,dc=univen... m
974 cn=10.200.26.3,cn=aRecord,cn=temporary,cn=univen... d
975 cn=52:54:00:9b:e5:01,cn=mac,cn=temporary,cn=univ... d
-------- 8< ----------

My observations are:
* The removal of temporary should be taken place after the creation of the UDM objects
* There is one temporary object which is not deleted
Comment 8 Florian Best univentionstaff 2016-08-26 15:06:39 CEST
*** Bug 42150 has been marked as a duplicate of this bug. ***
Comment 9 Florian Best univentionstaff 2016-09-20 13:59:34 CEST
*** Bug 42444 has been marked as a duplicate of this bug. ***
Comment 10 Florian Best univentionstaff 2016-09-22 19:19:15 CEST
*** Bug 7337 has been marked as a duplicate of this bug. ***
Comment 11 Florian Best univentionstaff 2016-09-24 16:12:13 CEST
*** Bug 18339 has been marked as a duplicate of this bug. ***
Comment 12 Florian Best univentionstaff 2016-09-24 16:12:38 CEST
*** Bug 20631 has been marked as a duplicate of this bug. ***
Comment 13 Florian Best univentionstaff 2016-10-21 14:05:39 CEST
*** Bug 26221 has been marked as a duplicate of this bug. ***
Comment 14 Florian Best univentionstaff 2016-10-21 14:05:59 CEST
*** Bug 23868 has been marked as a duplicate of this bug. ***
Comment 15 Florian Best univentionstaff 2016-10-21 14:34:18 CEST
*** Bug 31416 has been marked as a duplicate of this bug. ***
Comment 16 Jan Christoph Ebersbach univentionstaff 2017-01-04 11:55:24 CET
Again reported at Ticket #2017010321002241.
Comment 17 Florian Best univentionstaff 2017-02-10 09:55:17 CET
*** Bug 12125 has been marked as a duplicate of this bug. ***
Comment 18 Daniel Orrego univentionstaff 2017-04-24 10:58:55 CEST
Hi, 

Is this totally harmless bug or should one clean up those locks manually? or simply let them there and wait for an errata? For example, what will happen if one hast to re-join windows clients? 

We are also having a growing amount of locks in LDAP (Ticket#2017042421000191), after a migration from Samba3+LDAP to UCS 4.1-current. It's a bit more like in comment 7 (or Bug 31414)
Comment 19 Florian Best univentionstaff 2017-07-08 13:34:47 CEST
*** Bug 29684 has been marked as a duplicate of this bug. ***
Comment 20 Florian Best univentionstaff 2017-07-08 13:34:47 CEST
*** Bug 28632 has been marked as a duplicate of this bug. ***
Comment 21 Florian Best univentionstaff 2017-07-08 13:34:48 CEST
*** Bug 35834 has been marked as a duplicate of this bug. ***
Comment 22 Philipp Hahn univentionstaff 2017-09-15 18:08:24 CEST
Task #6732 UCS Technical Training

Tried to create user with wrong email domain
Creation was denied.
Seconds attempt failed because of remaining lock objects for UID(?)
Third attempt succeeded.
Comment 23 Florian Best univentionstaff 2017-11-21 12:18:53 CET
Reported again:
"""
Randfehler: Wenn wir obigen Befehl wiederholen kommen zwei andere Fehlermeldungen:

E: Object exists: (uid) : jsecord
The mail address is already in use.

bis wir nach dem dritten Aufruf wieder zur ersten kommen. Auch hier würden wir erwarten, dass die Fehlermeldungen konsistent sind.
"""
Comment 24 Sönke Schwardt-Krummrich univentionstaff 2018-09-10 22:14:17 CEST
Noticed again in UCS@school environments:
when creating a user, the lock object for the SID is not being removed after successful creation 
→ lots of lock objects are piling up and making e.g. the UCS join slow
Comment 25 Sönke Schwardt-Krummrich univentionstaff 2018-09-10 22:16:26 CEST
(In reply to Florian Best from comment #1)
> Created attachment 7779 [details]
> patch
> (needs rebase with svn r74405)

We should discuss this patch, before applying it.
Comment 26 Nico Stöckigt univentionstaff 2018-12-05 16:06:55 CET
With UCS 4.3-2 errata305 this is still the case, customer reported the daily creation of new tempory
Comment 27 Stefan Gohmann univentionstaff 2019-01-03 07:22:26 CET
This issue has been filled against UCS 4.1. The maintenance with bug and security fixes for UCS 4.1 has ended on 5st of April 2018.

Customers still on UCS 4.1 are encouraged to update to UCS 4.3. Please contact
your partner or Univention for any questions.

If this issue still occurs in newer UCS versions, please use "Clone this bug" or simply reopen the issue. In this case please provide detailed information on how this issue is affecting you.
Comment 28 Florian Best univentionstaff 2019-05-09 20:06:33 CEST
Created attachment 10017 [details]
patch (git:fbest/41711-fix-udm-locking)

Updated and enhanced the patch for UCS 4.4. Still only some small TODO's.