Univention Bugzilla – Bug 50630
ADC sync_from_ucs: Group members are replaced as whole instead of applying a diff
Last modified: 2020-04-15 18:06:54 CEST
The AD-Connector replaces all group members by replacing the whole members with the new+current members when syncing from UCS to AD: services/univention-s4-connector/modules/univention/s4connector/s4/__init__.py 1528 » def group_members_sync_from_ucs(self, key, object): … 1696 » » » » self.lo_s4.lo.modify_s(compatible_modstring(object['dn']), [(ldap.MOD_REPLACE, 'member', modlist_members)]) Instead we should remove all members with ldap.MOD_DELETE and add all new members with ldap.MOD_ADD. The code is prone to race conditions if there are changes inbetween on AD side: causing a loss of group members. Especially in large environments with a lot of group members this might have an performance impact. +++ This bug was initially created as a clone of Bug #50629 +++
Created attachment 10262 [details] patch? (git:fbest/50630-group-member-sync-from-ucs) Here is an untested patch.
The customer installed the patch and now gets the following traceback, when he adds a user to a group (via UCS): 08.01.2020 10:52:28,946 LDAP (WARNING): sync failed, saved as rejected 08.01.2020 10:52:28,946 LDAP (WARNING): Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/univention/connector/__init__.py", line 785, in __sync_file_from_ucs if ((old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, unicode(old_dn, 'utf8'))) or (not old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, old_dn))): File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 2645, in sync_from_ucs f(self, property_type, object) File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 176, in group_members_sync_from_ucs return connector.group_members_sync_from_ucs(key, object) File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 1806, in group_members_sync_from_ucs self.lo_s4.lo.modify_s(compatible_modstring(object['dn']), [(ldap.MOD_ADD, 'member', list(map(compatible_modstring, add_members)))]) AttributeError: ad instance has no attribute 'lo_s4'
Oh, replace "self.lo_s4" with "self.lo_ad" in the patch!
There is a PR for this bug: https://github.com/univention/univention-corporate-server/pull/16
Hi Daniel The PR is only geared towards Florian's branch (which has that typo he has mentioned). This I've not updated this ticket since I'd like help him in getting his patch properly into shape for inclusion as I'm definitely interested in getting this upstream. Regards Mathieu
After updating the UCS version to 4.4-3 the patch throws a traceback when a user is removed from a group. 04.02.2020 10:09:18.505 LDAP (PROCESS): sync from ucs: [ group] [ modify] cn=rg5_systemadministratoren,cn=groups,DC=ad,DC=schein,DC=ig 04.02.2020 10:09:18.544 LDAP (WARNING): sync failed, saved as rejected 04.02.2020 10:09:18.545 LDAP (WARNING): Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/univention/connector/__init__.py", line 785, in __sync_file_from_ucs if ((old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, unicode(old_dn, 'utf8'))) or (not old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, old_dn))): File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 2645, in sync_from_ucs f(self, property_type, object) File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 176, in group_members_sync_from_ucs return connector.group_members_sync_from_ucs(key, object) File "/usr/lib/pymodules/python2.7/univention/connector/ad/__init__.py", line 1808, in group_members_sync_from_ucs self.lo_ad.lo.modify_s(compatible_modstring(object['dn']), [(ldap.MOD_DEL, 'member', list(map(compatible_modstring, del_members)))]) AttributeError: 'module' object has no attribute 'MOD_DEL'
Are you refering to an installation with patch from Comment 1 applied? There's another typo in the patch, it's ldap.MOD_DELETE instead of ldap.MOD_DEL.
Successful build Package: univention-ad-connector Version: 13.0.0-31A~4.4.0.202003050945 Branch: ucs_4.4-0 Scope: errata4.4-3 cc53c886f8 Bug #50630: yaml 25b3ba09dd Bug #50630: Diff group members in ad-connector sync I fixed the typos in the patch and merged it. The AD-Connector tests pass and the list of group members is not replaced as a whole any more.
OK: ADD/DELETE instead of REPLACE OK: jenkins tests OK: yaml -> verified
http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-3/job/ADConnectorMultiEnv/Version=s4connector-w2k8r2-german/lastCompletedBuild/testReport/ * http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-3/job/ADConnectorMultiEnv/Version=s4connector-w2k8r2-german/24/testReport/55_adconnector/101sync_initial_membership_ad_to_ucs/admember237/ On the other hand, this test setup is *very* new and special, because it has S4-Connector *and* AD-Connector installed (for Bug #50492), and three S4-Connector related tests also failed, so it may simply be flaky. Please note that the term "admember237" is wrong, it was just a typo in the new test-scenario file. It's actually master237.
I've run this test several times on my VMs and at first it consistently failed. Then all of a sudden, it works, consistently. The funny thing is: Neither the test-user nor the test-groups get synchronized bei the S4-Connector to Samba/AD, so the "interference"/"timing" theory is ruled out to. I've just installed the ADC, configured SSL for the communication, installed ucs-test-adconnector and run the 101* test case individually. Crazy Heisenbug.
The test case in question is stable since two runs: * http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-3/job/ADConnectorMultiEnv/Version=s4connector-w2k8r2-german/lastCompletedBuild/testReport/55_adconnector/101sync_initial_membership_ad_to_ucs/history/ Additionally one of the other 4 S4-Connector related test cases could be explained by Bug #50944. So I'd assume that this Bug is not causing any regression.
(In reply to Arvid Requate from comment #12) > The test case in question is stable since two runs: > > * > http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-3/job/ > ADConnectorMultiEnv/Version=s4connector-w2k8r2-german/lastCompletedBuild/ > testReport/55_adconnector/101sync_initial_membership_ad_to_ucs/history/ > > Additionally one of the other 4 S4-Connector related test cases could be > explained by Bug #50944. > > So I'd assume that this Bug is not causing any regression. -> OK verified
<http://errata.software-univention.de/ucs/4.4/494.html>