Bug 46682 - Traceback with cross-school users after being removed from a school
Traceback with cross-school users after being removed from a school
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3-0-errata
Assigned To: Arvid Requate
Felix Botner
: 33466 47104 (view as bug list)
Depends on: 25709
Blocks: 46692 46971 47104 47636
  Show dependency treegraph
Reported: 2018-03-16 13:22 CET by Sönke Schwardt-Krummrich
Modified: 2018-08-23 12:06 CEST (History)
7 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 3: A User would likely not purchase the product
User Pain: 0.171
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Waiting Support: Yes
Ticket number: 2018031621000473
Bug group (optional):
Max CVSS v3 score:

skip_object_memberships_sync_to_ucs_if_group_syncmode_write.patch (1.04 KB, patch)
2018-03-21 22:01 CET, Arvid Requate
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sönke Schwardt-Krummrich univentionstaff 2018-03-16 13:22:00 CET
A school customer could now observe several times that the S4-Connector reproducibly throws tracebacks. It occurs in the following scenario:

A teacher is *only* at school1 and is temporarily set up as a cross-school user account for school "school1" and "school2". For this purpose, "school1" and "school2" are correctly entered in the user's LDAP attribute "ucsschoolSchool" and the user is additionally included in the groups "lehrer-school2" and "domain users school2". The teacher is then correctly replicated to the school2 slave and transferred to the AD via the S4 connector.
2 days later the user was removed from "school2" and the corresponding groups "lehrer-school2" and "domain users school2". This is said to have worked and the user has been correctly removed from LDAP and AD from the groups and the user object itself.
During the night the group "Domain Users school1" was modified. Since all groups "Domain User $SCHOOL" and "lehrer-$SCHOOL" are replicated to all schools, this change also arrived at the school DC dcschool2. The S4 connector has thrown the following traceback:

22.02.2018 07:15:16,924 LDAP        (WARNING): group_members_sync_from_ucs: failed to sync members: (cn=domain users school1,cn=groups,ou=school1,DC=schule,DC=customer,DC=de,[(2, 'member', ['cn=someteacher,cn=lehrer,cn=users,ou=school1,dc=schule,dc=customer,dc=de'])])
22.02.2018 07:15:16,930 LDAP        (WARNING): sync failed, saved as rejected
22.02.2018 07:15:16,967 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/s4connector/__init__.py", line 897, in __sync_file_from_ucs
    if ((old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, unicode(old_dn, 'utf8'), old, new)) or (not old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, old_dn, old, new))):
  File "/usr/lib/pymodules/python2.7/univention/s4connector/s4/__init__.py", line 2720, in sync_from_ucs
    f(self, property_type, object)
  File "/usr/lib/pymodules/python2.7/univention/s4connector/s4/__init__.py", line 79, in group_members_sync_from_ucs
    return s4connector.group_members_sync_from_ucs(key, object)
  File "/usr/lib/pymodules/python2.7/univention/s4connector/s4/__init__.py", line 1812, in group_members_sync_from_ucs
    self.lo_s4.lo.modify_s(compatible_modstring(object['dn']), [(ldap.MOD_REPLACE, 'member', modlist_members)])
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 364, in modify_s
    return self.result(msgid,all=1,timeout=self.timeout)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 465, in result
    resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 469, in result2
    resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 476, in result3
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 483, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 106, in _ldap_call
    result = func(*args,**kwargs)
NO_SUCH_OBJECT: {'info': '00002030: Unable to find GUID for DN cn=someteacher,cn=lehrer,cn=users,ou=school1,dc=schule,dc=customer,dc=de\n', 'desc': 'No such object'}
Comment 1 Arvid Requate univentionstaff 2018-03-21 19:33:23 CET
I cannot reproduce it, neither in UCS 4.2-3 nor UCS 4.3-0, trying three different approaches of removal of the teacher account from "school2":

A) Remove school2 from ucsschoolSchool and the user from the groups
   in one step via UMC
B) First remove school2 from ucsschoolSchool via UMC, save.
   Then remove user from the groups in separate step, save.
C) First remove user from groups via UMC, save.
   Then school2 from ucsschoolSchool in separate step, save.

No success.
Comment 2 Arvid Requate univentionstaff 2018-03-21 22:00:54 CET
It would be very interesting to know, if the user object already had been removed in Samba/AD at that point, but Samba failed to also remove the group membership. That's the only scenario I can currently think of. If we cannot get more information we might have to add some debugging code which runs s4search on the problematic member object.

Anyway, fact is, that Samba cannot remove the group membership. Maybe a samba-tool dbcheck would be required to fix this. We probably cannot fix that in the S4-Connector "group_members_sync_from_ucs". But what we should do, is to avoid synchronizing the group memberships back, especially since we have connector/s4/mapping/group/syncmode=write !

Currently I can think of two ways to do this:

1) *If* the user object is already removed in Samba/AD, then why do group memberships synchronized back? Maybe we could skip that.

2) The mapping still calls "object_memberships_sync_to_ucs" even though the group syncmode is "write". We sh/could remove that call in this configuration. See attached patch sketch.
Comment 3 Arvid Requate univentionstaff 2018-03-21 22:01:23 CET
Created attachment 9484 [details]
Comment 4 Sönke Schwardt-Krummrich univentionstaff 2018-03-23 12:10:00 CET
Switching to "NEW" → otherwise the bug would not show up in several reporting tools
Comment 5 Arvid Requate univentionstaff 2018-04-26 19:41:56 CEST
3e9db0b31a | skip object_memberships_sync_to_ucs if group syncmode is write
0d1a2affb4 | Advisory
Comment 6 Quality Assurance univentionstaff 2018-05-04 16:43:04 CEST
--- mirror/ftp/4.3/unmaintained/component/4.3-0-errata/source/univention-s4-connector_12.0.2-10A~
+++ apt/ucs_4.3-0-errata4.3-0/source/univention-s4-connector_12.0.2-11A~
@@ -1,6 +1,11 @@
-12.0.2-10A~ [Mon, 16 Apr 2018 13:12:03 +0200] Univention builddaemon <buildd@univention.de>:
+12.0.2-11A~ [Thu, 26 Apr 2018 19:33:53 +0200] Univention builddaemon <buildd@univention.de>:
   * UCS auto build. No patches were applied to the original source package
+12.0.2-11 [Thu, 26 Apr 2018 19:27:48 +0200] Arvid Requate <requate@univention.de>:
+  * Bug #46682: skip object_memberships_sync_to_ucs
+    if group syncmode is write
 12.0.2-10 [Mon, 16 Apr 2018 13:10:35 +0200] Felix Botner <botner@univention.de>:
Comment 7 Stefan Gohmann univentionstaff 2018-05-07 16:38:05 CEST
*** Bug 33466 has been marked as a duplicate of this bug. ***
Comment 8 Arvid Requate univentionstaff 2018-05-25 13:34:48 CEST
QA for Bug 46971 showed that my initial patch was not effective.
Comment 9 Arvid Requate univentionstaff 2018-05-30 15:37:41 CEST
*** Bug 47104 has been marked as a duplicate of this bug. ***
Comment 10 Arvid Requate univentionstaff 2018-06-04 14:00:26 CEST
Commits cherrypicked from 4.2-4: 147232dc33^..035dbabe63 and 7b9fef72c9.

Two patch hunks ignored from commit 92f8e177e9 because the target code has been removed in 4.3-0 (due to commit 01447fb6ce for Bug #47013)

774394cabe | Advisory package version
0af5140b73 | Changelog & Advisory
cef3856cf9 | Fix code comment
d7ad96f5f5 | Fix traceback
b8be095cfc | Code cleanup: Improve readability
1005980bc4 | Code cleanup: Improve readability
Comment 11 Felix Botner univentionstaff 2018-06-04 15:48:02 CEST
OK - univention-s4-connector
OK - yaml