Univention Bugzilla – Bug 34355
Modify and move objects in a short amount of time
Last modified: 2014-05-26 15:40:24 CEST
Ticket #2014012921002092 If you modify and move objects in a really short time it is possible that the listener does not find the original object for building new + old for the listener. For example: udm computers/windows modify --dn "$dn" --set description=foo1 udm computers/windows move --dn "$dn" --position ou=computers,ou=Z,$ldap_base This results in the following transaction: 1062029 cn=client1,cn=computers,$ldap_base m 1062030 cn=client1,cn=computers,$ldap_base r 1062031 cn=client1,ou=computers,ou=Z,$ldap_base a The listener reads 1062029 and searches for the object. If the object is already moved, the listener would find the object and thinks it was deleted. That's a problem for some modules for example the s4 connector. In the test environment the following workaround seems to "fix" the problem: ucr set listener/uniquemember/skip=no listener/memberuid/skip=no mv /var/lib/univention-directory-listener/handlers/s4-connector /var/lib/univention-directory-listener/handlers/aas4-connector mv /usr/lib/univention-directory-listener/system/s4-connector.py /usr/lib/univention-directory-listener/system/aas4-connector.py sed -i 's|s4-connector|aas4-connector|' /usr/lib/univention-directory-listener/system/aas4-connector.py
(In reply to Stefan Gohmann from comment #0) > In the test environment the following workaround seems to "fix" the problem: > [...] Sure this helps only for the s4 connector. The issue is still valid with other listener modules and when the listener is stopped at the time the modification was made (modify and move).
As discussed with Alex, we should switch from DN to entryUUID. But we have to be careful with updates, e.g. - new listener → old notifier - old listener → new notiifier
I've added a test case for this issue (r49006): 10_ldap/49replication_modrdn_with_previous_modification Please let me know if you need more or different test cases.
Search by entryUUID basically works, but there is this scenario: modify uid=old,cn=old,... modrdn uid=new,cn=new,... this would get translated to: m uid=old,cn=old,... r uid=old,cn=old,... a uid=new,cn=new,... old behavior: command="m" dn="uid=old,cn=old,..." old={'desc':['old'],'entryDN':['uid=old,cn=old,...'],...} new={'desc':['new'],'entryDN':['uid=old,cn=old,...'],...} command="r" dn="uid=old,cn=old,..." old={'desc':['new'],'uid':['old'],'entryDN':['uid=old,cn=old,...'],...} new=None command="a" dn="uid=new,cn=new,..." old=None new={'desc':['new'],'uid':['new'],'entryDN':['uid=new,cn=new,...'],...} with entryUUID the new 'dn' and new 'uid' already gets visible in the first modify: command="m" dn="uid=old,cn=old,..." old={'desc':['old'],'uid':['old'],'entryDN':['uid=old,cn=old,...'],...} new={'desc':['new'],'uid':['new'],'entryDN':['uid=new,cn=new,...'],...} command="r" dn="uid=old,cn=old,..." old=??? new=None command="a" dn="uid=new,cn=new,..." old=None new={'desc':['new'],'uid':['new'],'entryDN':['uid=new,cn=new,...'],...} Option 1: re-construct the missed change by filtering out the change in DN and just submit the change in the other attributes to the handler. The move will be handled later. Option 2: Create a artificial 'r'/'a' before the 'm', so listeners will already get the new DN in case they do a LDAP search for the new DN on their own. TBC...
r49564 | Bug #34355 Listener: Fix move after modify r49563 | Bug #34355 Listener: Track move after modify by UUID r49562 | Bug #34355 Listener: Add cache_entry_[sg]et1 r49561 | Bug #34355 Listener: Move error message into function r49560 | Bug #34355 Listener: Introduce STR[N]EQ macros r49559 | Bug #34355 Listener: Set NULL on free r49558 | Bug #34355 Listener: Cleanup cache_dump_entry r49557 | Bug #34355 Listener: Drop NotifierId from cache_get() r49556 | Bug #34355 Listener: Add test cases r49555 | Bug #34355 Listener: Document r=modrdn r49554 | Bug #34355 Listener: Update copyright univention-directory-listener_8.0.0-7.218.201404232005 branches/ucs-3.2/ucs-3.2-1/doc/errata/staging/2014-04-23_univention-directory-listener.yaml r49565 | Bug #34355 Listener: YAML for move after modify The listener now uses the entryUUID of objects to track moves. Modifications to moved objects are delayed until the object reaches it's final destination; then all aggregated changes are applied at once. Consecutive moves are executed one-for-one to handle the case, where objects are moved to locations already inherited by other objects. See the files in the directory doc.34355/ for more details and test cases.
Please have a look at the jenkins tests: jenkins.knut.univention.de:8080/job/UCS 3.2 Autotest MultiEnv/358/testReport/ For example the following case failed: 00_base/95rename_administrator (In reply to Philipp Hahn from comment #5) > r49564 | Bug #34355 Listener: Fix move after modify > r49563 | Bug #34355 Listener: Track move after modify by UUID > r49562 | Bug #34355 Listener: Add cache_entry_[sg]et1 > r49561 | Bug #34355 Listener: Move error message into function > r49560 | Bug #34355 Listener: Introduce STR[N]EQ macros Hm, is this really better? I don't like the usage of macros for this case because you must always check what the macro does. If we have to change this, we should rather use "if(!strcmp())" and "if(strcmp())". > r49559 | Bug #34355 Listener: Set NULL on free > r49558 | Bug #34355 Listener: Cleanup cache_dump_entry > r49557 | Bug #34355 Listener: Drop NotifierId from cache_get() > r49556 | Bug #34355 Listener: Add test cases Please file a new bug against ucs-test. I think these test cases should be moved to ucs-test. > r49555 | Bug #34355 Listener: Document r=modrdn > r49554 | Bug #34355 Listener: Update copyright
(In reply to Stefan Gohmann from comment #6) > Please have a look at the jenkins tests: > jenkins.knut.univention.de:8080/job/UCS 3.2 Autotest > MultiEnv/358/testReport/ > > For example the following case failed: > 00_base/95rename_administrator The listener module "well-known-sid-name-mapping" is buggy: it wrongly assumes that "uid" is unique. The new listener now first does a "modrdn", which keeps the old "uid=Administrator" and adds an additional "uid=$RANDOM". The listener module then assumes that on the "move_to" part the uid already contains only the new uid, but only fetches uid[0]='"Administrator" and thus does not trigger the code to set the UCRV. See Bug #33890 for some background and Bug #34742 for the real problem. > > r49560 | Bug #34355 Listener: Introduce STR[N]EQ macros > > Hm, is this really better? I don't like the usage of macros for this case > because you must always check what the macro does. > If we have to change > this, we should rather use "if(!strcmp())" and "if(strcmp())". That's why naming is important and STREQ should be obvious. With strcmp() you have to parse the full statement to decide if there a "!= 0" or a "== 0" following, which makes it explicit if you're testing for equality. The !-form on the other hand completely hides this and using !strcmp() for equality looks very strange. > > r49556 | Bug #34355 Listener: Add test cases > Please file a new bug against ucs-test. I think these test cases should be > moved to ucs-test. → Bug #34700 r49758 | Bug #34355 Listener: fix memory leak r49759 | Bug #34355 Listener: Add test cases 2 8.0.0-8.219.201405030139 r49761 | Bug #34355 Listener: YAML for move after modify > Please have a look at the jenkins tests: Waiting for next round...
(In reply to Philipp Hahn from comment #7) > (In reply to Stefan Gohmann from comment #6) > > Please have a look at the jenkins tests: > Waiting for next round... The case for well-known-sids was successfully fixed by my change through Bug #34742. But <http://jenkins.knut.univention.de:8080/view/Autotest/job/UCS%203.2%20Autotest%20MultiEnv/368/SambaVersion=s4,Systemrolle=slave/testReport/10_ldap/45replication_modrdn/test/> failed on S4-Slave. Running the same on my xen12+Slave_as_VM works fine. The run did non succeed on S4-{Master,Backup}; looks like an EC2 Problem.
r49800 | Bug #34355 Listener: Remove old values on RDN change univention-directory-listener_8.0.0-9.220.201405051622 OK: Both old and new version of "well-known-sid-name-mapping" run fine with /usr/share/ucs-test/00_base/95rename_administrator OK: run-parts --verbose --regex ^test.* doc.34355
Fixed a bug in handling attributes containing '\0', but UDM is buggy: Bug #34749 r49835 | Bug #34355 Listener: binary attribute fix r49834 | Bug #34355 Listener: test non-ASCII rename r49833 | Bug #34355 Listener: Revert STR[N]EQ macros univention-directory-listener_8.0.0-10.221.201405061116 YAML-update: TODO
Code Review (svn diff -r4769):OK YAML: OK (updated version number r49909) Missing: jenkins and manual tests
Tests: OK
https://forge.univention.org/bugzilla/show_bug.cgi?id=34355
http://errata.software-univention.de/ucs/3.2/104.html