Bug 37259 - GPO rejects SINGLE-VALUE attribute attribute specified more than once versionNumber, gPCUserExtensionNames, gPCMachineExtensionNames
GPO rejects SINGLE-VALUE attribute attribute specified more than once version...
Product: UCS
Classification: Unclassified
Component: S4 Connector
UCS 4.0
Other Linux
: P5 normal (vote)
: UCS 4.0-2-errata
Assigned To: Arvid Requate
Felix Botner
: 38056 (view as bug list)
Depends on:
Blocks: 38813 39365
  Show dependency treegraph
Reported: 2014-12-09 15:10 CET by Tim Petersen
Modified: 2015-09-18 14:43 CEST (History)
7 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:
requate: Patch_Available+

connector.s4.log (7.56 KB, text/plain)
2015-04-30 16:19 CEST, Janis Meybohm
ldap.ldif (925 bytes, text/plain)
2015-04-30 16:20 CEST, Janis Meybohm
s4.ldif (1.29 KB, text/plain)
2015-04-30 16:20 CEST, Janis Meybohm
single_value.patch (5.87 KB, patch)
2015-04-30 16:56 CEST, Arvid Requate
Details | Diff
produce_a_resolvable_conflict_like_bug_37259.sh (2.15 KB, text/plain)
2015-06-04 16:43 CEST, Arvid Requate
produce_a_unresolvable_conflict_like_bug_37259.sh (2.90 KB, text/plain)
2015-06-04 16:46 CEST, Arvid Requate

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Petersen univentionstaff 2014-12-09 15:10:11 CET
I saw rejects on two customer systems till now ... (2014120121000455, 2014120921000253).

GPO's get rejected with the following Traceback:

09.12.2014 06:25:38,410 LDAP        (PROCESS): sync from ucs:   Resync rejected file: /var/lib/univention-connector/s4/1418057385.014873
09.12.2014 06:25:38,411 LDAP        (PROCESS): sync from ucs: [         msGPO] [    modify] cn={322c2f1e-721d-4ef2-9240-fb72b9d04b63},cn=policies,cn=system,dc=domain,dc=de
09.12.2014 06:25:38,420 LDAP        (ERROR  ): sync_from_ucs: traceback during modify object: cn={322c2f1e-721d-4ef2-9240-fb72b9d04b63},cn=policies,cn=system,dc=domain,dc=de
09.12.2014 06:25:38,420 LDAP        (ERROR  ): sync_from_ucs: traceback due to modlist: [(0, 'versionNumber', set([u'524288']))]
09.12.2014 06:25:38,428 LDAP        (WARNING): sync failed, saved as rejected
09.12.2014 06:25:38,430 LDAP        (WARNING): Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/univention/s4connector/__init__.py", line 785, in __sync_file_from_ucs
    or (not old_dn and not self.sync_from_ucs(key, object, premapped_ucs_dn, old_dn, old, new))):
  File "/usr/lib/pymodules/python2.6/univention/s4connector/s4/__init__.py", line 2505, in sync_from_ucs
    self.lo_s4.lo.modify_ext_s(compatible_modstring(object['dn']), compatible_modlist(modlist), serverctrls=self.serverctrls_for_add_and_modify)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 808, in modify_ext_s
    return self._apply_method_s(SimpleLDAPObject.modify_ext_s,*args,**kwargs)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 766, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 295, in modify_ext_s
    return self.result(msgid,all=1,timeout=self.timeout)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 422, in result
    res_type,res_data,res_msgid = self.result2(msgid,all,timeout)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 426, in result2
    res_type, res_data, res_msgid, srv_ctrls = self.result3(msgid,all,timeout)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 432, in result3
    ldap_result = self._ldap_call(self._l.result3,msgid,all,timeout)
  File "/usr/lib/python2.6/dist-packages/ldap/ldapobject.py", line 96, in _ldap_call
    result = func(*args,**kwargs)
TYPE_OR_VALUE_EXISTS: {'info': '0000200D: SINGLE-VALUE attribute versionNumber on CN={322C2F1E-721D-4EF2-9240-FB72B9D04B63},CN=Policies,CN=System,DC=domain,DC=de specified more than once', 'desc': 'Type or value exists'}

The object itself seems already equal in ldap and s4.
Comment 1 Tim Petersen univentionstaff 2014-12-09 15:10:32 CET
I wasn't able to reproduce it till now
Comment 2 Tim Petersen univentionstaff 2014-12-16 07:18:36 CET
First seen in 3.2-4 - I suppose it was introduced then
Comment 3 Tim Petersen univentionstaff 2015-03-11 11:24:14 CET
Seen again - School environment on a school slave: 2015030621000087
Comment 4 Tim Petersen univentionstaff 2015-03-11 11:24:45 CET
(In reply to Tim Petersen from comment #3)
> Seen again - School environment on a school slave: 2015030621000087

Comment 5 Janis Meybohm univentionstaff 2015-04-30 16:11:57 CEST
*** Bug 38056 has been marked as a duplicate of this bug. ***
Comment 6 Janis Meybohm univentionstaff 2015-04-30 16:19:50 CEST
Created attachment 6864 [details]

Bug 38056 reported this for Ticket#2014112021000242

Reported again via Ticket#2015042921000345 for versionNumber and gPCMachineExtensionNames

Don't know how it happened but I just noticed the versionNumber traceback in one of my test environments as well. Debuglevel 4 log attached
Comment 7 Janis Meybohm univentionstaff 2015-04-30 16:20:09 CEST
Created attachment 6865 [details]
Comment 8 Janis Meybohm univentionstaff 2015-04-30 16:20:19 CEST
Created attachment 6866 [details]
Comment 9 Arvid Requate univentionstaff 2015-04-30 16:56:35 CEST
Created attachment 6867 [details]

Yes, big fail of some unknown author. Patch attached fixes this:

All of those attributes need to be marked single_value=True in the mapping file. Maybe this should be the default in the S4-Connector to begin with, because most attributes in the mapping are marked as such.

For the record: Whenever we add a new attribute the the mapping we need to check this:

univention-s4search -b "cn=Schema,cn=Configuration,$samba4_ldap_base" \
   lDAPDisplayName=gPCMachineExtensionNames isSingleValued
Comment 10 Janis Meybohm univentionstaff 2015-04-30 17:06:42 CEST
(In reply to Arvid Requate from comment #9)
> Created attachment 6867 [details]
> single_value.patch
> Yes, big fail of some unknown author. Patch attached fixes this:
Well, well. Patch from some unknown author fixes fail of some unknown author at least in my test env. :-)

4.0-1-errata as this will hit all AD environments sooner or later.
Comment 11 Arvid Requate univentionstaff 2015-05-06 12:34:56 CEST
We need to take care here that the S4-Connector doesn't overwrite attribute values in Samba4 which changed (e.g. via RSAT tools) in the time interval between reject and <now>.

The issue can bee seen directly in the connector-s4.log of Comment 6 and the ldif attached to Comment 7:
* The reject want's to write versionNumber=4 to Samba4
* Samba4 already has versionNumber=24
* OpenLDAP also has msGPOVersionNumber=24

Interestingly, OpenLDAP and Samba4 are in Sync already (because the last change has been in Samba4 and the sync to OpenLDAP works), so the reject must not be applied in this case!

We need to write another script to fix this.

Maybe we can even learn a general point for the S4-Connector from this: When treating rejects, the S4-Connector (c|sh)ould check if the "old" value is still valid in the source directory. If not, then the change could be ignored. I created enhancement Bug 38450 for that.
Comment 12 Arvid Requate univentionstaff 2015-05-07 20:02:10 CEST
I attached a reproducer and a patch for Comment 11 to Bug #38450.
Comment 13 Dirk Ahrnke 2015-05-08 14:15:02 CEST
The solution for this bug is targeted for 4.0-2 errata
Comment 11 mentions UCS 3.2-4
In http://forum.univention.de/viewtopic.php?f=48&t=3835&p=14306#p14306 another customer asks if there is/will be a solution for UCS 3.2
Comment 14 Arvid Requate univentionstaff 2015-05-11 11:06:25 CEST
Technically I see no issues in backporting the patch once it's done for 4.0-2.
Comment 15 Janis Meybohm univentionstaff 2015-06-01 14:55:27 CEST
Reported again by Ticket#2015052921000512
Comment 16 Arvid Requate univentionstaff 2015-06-03 19:33:54 CEST
In postinst we now call a script "remove_obsolete_gpo_and_wmi_rejects" which looks at each UCS rejected change file. If the change affects one of the attributes which were affected by this bug, then it compares the *current* values of those attributes in OpenLDAP with those in Samba4. If they already are "in sync", then the reject is removed.

Advisory: 2015-06-03-univention-s4-connector.yaml

This still needs some testing.
Comment 17 Arvid Requate univentionstaff 2015-06-04 16:43:00 CEST
Created attachment 6941 [details]

Reproducer script.
Comment 18 Arvid Requate univentionstaff 2015-06-04 16:46:08 CEST
Created attachment 6942 [details]

A different reproducer, which creates a conflict that cannot be resolved by the adjust_obsolete_gpo_and_wmi_rejects script which is the improved version of the remove_obsolete_gpo_and_wmi_rejects script mentioned in Comment 16.
Comment 19 Arvid Requate univentionstaff 2015-06-04 17:04:49 CEST
The remove_obsolete_gpo_and_wmi_rejects script has benn improved and renamed to adjust_obsolete_gpo_and_wmi_rejects. It now doesn't remove the reject but only fix the attribute value in question (if it can be fixed automatically). That way, the reject disappears the next time the S4-Connector is started. It could contain additional changes which should not been thrown away.

The two reproducer scripts above can be used to produce rejects that are (a) fixable and (b) not fixable by this script. The not fixable ones are left as they are. (Theoretically one could do something with timestamps but this is a lot of additional work and not really required here. The (b) case should not appear in the wild.).

I added a ucs-test case to generally check that all Samba4 attributes used in the mapping are configured as single-value if they are declared as such in the Samba4/AD schema:

* 52_s4connector/402check_mapping_for_single_value_samba4_attributes
Comment 20 Janis Meybohm univentionstaff 2015-06-10 14:55:03 CEST

"adjust_obsolete_gpo_and_wmi_rejects" successfully adjusted ~30 rejects in customer environment.
Comment 21 Stefan Gohmann univentionstaff 2015-06-13 14:55:09 CEST
The test case 402check_mapping_for_single_value_samba4_attributes fails in jenkins:

*** BEGIN *** ['/usr/bin/python', '402check_mapping_for_single_value_samba4_attributes'] ***
*** 52_s4connector/402check_mapping_for_single_value_samba4_attributes *** S4-connector check the mapping for single-value Samba4 attributes ***
*** START TIME: 2015-06-13 04:21:15 ***
INFO: Checking if all Samba4 attributes in the S4-Connector mapping are properly declared as Single-Value
### FAIL ###
ERROR: Some single valued Samba4 attributes are not configured properly in the S4-Connector mapping.
###      ###
*** END TIME: 2015-06-13 04:21:16 ***
*** TEST DURATION (H:MM:SS.ms): 0:00:00.839408 ***
*** END *** 1 ***
Comment 22 Arvid Requate univentionstaff 2015-06-15 12:36:31 CEST
Ok, fixed a trivial bug in the test.
Comment 23 Felix Botner univentionstaff 2015-06-15 13:44:12 CEST
please add the post_attributes to the test.
Comment 24 Arvid Requate univentionstaff 2015-06-15 14:01:40 CEST
Comment 25 Felix Botner univentionstaff 2015-06-15 14:45:15 CEST
OK - ucs test
OK - update, resolve GPO and WMI SINGLE-VALUE rejects 
OK - missing single value config for GPO and WMI

Comment 26 Janek Walkenhorst univentionstaff 2015-06-17 18:16:18 CEST