Bug 41580 - objectClass is not removed from object when extended attribute wants it
objectClass is not removed from object when extended attribute wants it
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UDM - Extended Attributes
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1-2-errata
Assigned To: Florian Best
Philipp Hahn
:
: 21608 28145 29034 (view as bug list)
Depends on: 41207
Blocks:
  Show dependency treegraph
 
Reported: 2016-06-15 15:28 CEST by Florian Best
Modified: 2016-09-29 17:31 CEST (History)
8 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.429
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional): External feedback
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2016-06-15 15:28:24 CEST
Contining Bug #41207 comment 17:

(In reply to Philipp Hahn from comment #17)
> FORK: Bug #21608 2): currently the objectClass associated with an Extended
> Option can't be removed, because UDM has no logic to parse the
> "objectClass"es on load and to enable the associated options. That only
> works if at least one "attribute" of that objectClass is loaded. Thus
> neither UDM-UMC nor udm-cli show the option as being enabled after loading.
> Thus the option can't be deselected, thus the objectClass is not removed.
> 
> FAIL: LDAP-Schema-handling is incomplete due to
> multiple-alias-names-per-attribute:
>  # ldapsearch -LLLo ldif-wrap=no -x -b cn=Subschema -s base attributeTypes
> objectClasses | less
>  # univention-ldapsearch -LLLb cn=bqwds365ti,cn=groups,dc=phahn,dc=qa
> objectClass structuralObjectClass
> dn: cn=bqwds365ti,cn=groups,dc=phahn,dc=qa
> objectClass: posixGroup
> objectClass: sambaGroupMapping
> objectClass: top
> objectClass: univentionGroup
> objectClass: univentionFreeAttributes
> objectClass: univentionObject
> structuralObjectClass: posixGroup
> 
>  # ldapsearch -LLLo ldif-wrap=no -x -b cn=Subschema -s base objectClasses |
> grep posixGroup
> objectClasses: ( 1.3.6.1.1.1.2.2 NAME 'posixGroup' DESC 'Abstraction of a
> group of accounts' SUP top STRUCTURAL MUST ( cn $ gidNumber ) MAY (
> userPassword $ memberUid $ description ) )
> 
>  # ldapsearch -LLLo ldif-wrap=no -x -b cn=Subschema -s base attributeTypes |
> grep --word cn
> dn: cn=Subschema
> attributeTypes: ( 2.5.4.3 NAME ( 'cn' 'commonName' ) DESC 'RFC4519: common
> name(s) for which the entity is known by' SUP name )
> 
> self.oldattr() only contains 'cn', but not 'commonName'; or vis versa. Patch
> follows.
> 
> 
> FYI: There's a 2nd issue when two EAs are defined using the same OC, but
> only one using an option. In that case removing the dependent EA also
> removed the non-dependent EA. This is not nice, but IMHO preferable to only
> one EA being removed, while the 2nd EA survives and the EO is shown again
> next time, as one of the EAs still exists.
> 
> 
> OK: r70051 r70050 r70049 r70048
> 
> OK: dpkg-query -W python-univention-directory-manager #
> 11.0.3-10.1402.201606141456
> OK: dpkg-query -W python-univention # 9.0.1-3.161.201606091857
> 
> OK: errata-announce -V --only univention-directory-manager-modules.yaml
> OK: univention-directory-manager-modules.yaml
> 
> OK: errata-announce -V --only univention-python.yaml
> FIXED: univention-python.yaml
>  r70191 | Bug #41207 QA: Fix YAML

+++ This bug was initially created as a clone of Bug #41207 +++

On the Tab 'LDAP mapping' within LDAP/Extended Attributes, there is a Checkbox labeled 'Remove object class if the attribute is removed'. When it's checked and the Attribute is cleared/removed, the ObjectClass remains at the Object itself.

Happened here Ticket #2016050321000407
Comment 1 Florian Best univentionstaff 2016-06-16 01:49:22 CEST
RFC: I *maybe* just found another bug: If a extended attribute which doesn't depend on an option defines objectClass=foo the object class is also set if the attribute is not set. Should we adjust this behavior as well?
Comment 2 Florian Best univentionstaff 2016-06-16 01:51:04 CEST
(In reply to Florian Best from comment #1)
> RFC: I *maybe* just found another bug: If a extended attribute which doesn't
> depend on an option defines objectClass=foo the object class is also set if
> the attribute is not set. Should we adjust this behavior as well?

→ This is maybe the cause for comment #0
...> FYI: There's a 2nd issue when two EAs are defined using the same OC...
Comment 3 Florian Best univentionstaff 2016-06-21 09:58:44 CEST
*** Bug 28145 has been marked as a duplicate of this bug. ***
Comment 4 Florian Best univentionstaff 2016-06-21 11:30:06 CEST
(In reply to Philipp Hahn from comment #17)
> FORK: Bug #21608 2): currently the objectClass associated with an Extended
> Option can't be removed, because UDM has no logic to parse the
> "objectClass"es on load and to enable the associated options. That only
> works if at least one "attribute" of that objectClass is loaded. Thus
> neither UDM-UMC nor udm-cli show the option as being enabled after loading.
> Thus the option can't be deselected, thus the objectClass is not removed.
A generic implementation has been added into simpleLDAP which evaluates the set options when initializing a instance of 'object', object classes are added in _ldap_addlist() and _ldap_modlist() when options where enabled, object classes are removed in _ldap_modlist() if options gets removed, attributes belonging to the option are also removed in diff() if the option was deselected.

The following modules currently already use options, they have been adjusted to use this generic mechanism:
['computers/domaincontroller_backup', 'computers/domaincontroller_master', 'computers/domaincontroller_slave', 'computers/ipmanagedclient', 'computers/linux', 'computers/macos', 'computers/memberserver', 'computers/ubuntu', 'computers/windows', 'computers/windows_domaincontroller', 'container/dc', 'groups/group', 'settings/license', 'shares/share', 'users/user']

> FAIL: LDAP-Schema-handling is incomplete due to
> multiple-alias-names-per-attribute:
>  # ldapsearch -LLLo ldif-wrap=no -x -b cn=Subschema -s base attributeTypes
> objectClasses | less
>  # univention-ldapsearch -LLLb cn=bqwds365ti,cn=groups,dc=phahn,dc=qa
> objectClass structuralObjectClass
> dn: cn=bqwds365ti,cn=groups,dc=phahn,dc=qa
> objectClass: posixGroup
> objectClass: sambaGroupMapping
> objectClass: top
> objectClass: univentionGroup
> objectClass: univentionFreeAttributes
> objectClass: univentionObject
> structuralObjectClass: posixGroup
> 
>  # ldapsearch -LLLo ldif-wrap=no -x -b cn=Subschema -s base objectClasses |
> grep posixGroup
> objectClasses: ( 1.3.6.1.1.1.2.2 NAME 'posixGroup' DESC 'Abstraction of a
> group of accounts' SUP top STRUCTURAL MUST ( cn $ gidNumber ) MAY (
> userPassword $ memberUid $ description ) )
> 
>  # ldapsearch -LLLo ldif-wrap=no -x -b cn=Subschema -s base attributeTypes |
> grep --word cn
> dn: cn=Subschema
> attributeTypes: ( 2.5.4.3 NAME ( 'cn' 'commonName' ) DESC 'RFC4519: common
> name(s) for which the entity is known by' SUP name )
> 
> self.oldattr() only contains 'cn', but not 'commonName'; or vis versa. Patch
> follows.
Thank you for the patch. I used it as base for the modifications.
There are probably a lot more places where UDM breaks if aliases are used (e.g. the mapping).
Comment 5 Florian Best univentionstaff 2016-06-21 12:00:23 CEST
*** Bug 29034 has been marked as a duplicate of this bug. ***
Comment 6 Stefan Gohmann univentionstaff 2016-06-22 06:02:35 CEST
The Jenkins test setup environments failed with the following traceback:

Configure /usr/lib/univention-install/05univention-bind.inst
2016-06-21 17:03:47.086304567-04:00 (in joinscript_init)
Adding ZONE record "root@autotest227.local. 1 28800 7200 604800 10800 admember227.autotest227.local." to zone autotest227.local...
Traceback (most recent call last):
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 400, in <module>
    main()
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 375, in main
    add_zone(*args)
  File "/usr/share/univention-admin-tools/univention-dnsedit", line 327, in add_zone
    __MSG__:Configure 08univention-apache
__STEP__:6
Configure /usr/lib/univention-install/08univention-apache.inst
zone = forward_zone.object(co, lo, position)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/dns/forward_zone.py", line 246, in __init__
    univention.admin.handlers.simpleLdap.__init__(self, co, lo, position, dn, superordinate, attributes = attributes )
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/__init__.py", line 541, in __init__
    self.mapping = m.mapping
AttributeError: 'NoneType' object has no attribute 'mapping'
Comment 7 Florian Best univentionstaff 2016-06-22 09:47:13 CEST
(In reply to Stefan Gohmann from comment #6)
> The Jenkins test setup environments failed with the following traceback:
This has been fixed, I restarted the jenkins UCS 4.1-2-errata job.
Comment 8 Jens Thorp-Hansen univentionstaff 2016-06-27 13:49:07 CEST
happened again: Ticket#2016062121000443
Comment 9 Florian Best univentionstaff 2016-06-27 13:58:28 CEST
Ticket#2016062321000127
Comment 10 Florian Best univentionstaff 2016-06-27 14:08:06 CEST
univention-directory-manager-modules (11.0.3-20):
r70514 | Bug #41580: fixup svn r70447; make sure modules are loaded

r70503 | Bug #41580: use generic option parsing/composing for computers/

 * Remove unused variable "tmppos":
 tmppos=univention.admin.uldap.position(self.position.getDomain())
 → has no effect and is probably not for error handling. It could only raise a exception insufficientInformation(_("There was no LDAP base specified.")) if the LDAP base DN doesn't contain 'dc='.
 * property homePostalAddress "nowerdays" always use "postalAddress" syntax.
 * shift() and setPassword() are unused.
 
r70469 | Bug #41580: fixup svn r70452; Bug #29034: add options to module if not exists

r70452 | Bug #41580: Rmove dead code when evaluating options

 * self.has_key() already checks if the options are enabled
 * use set() syntax
 * adjust/remove some unnecessary type checking
 * add some stub functions
 
r70451 | Bug #41580: Remove attributes which are disabled by options

 * Attributes which are disabled by an option should be removed. The diff
 previously covered only changes while all attributes must be removed.
 
r70450 | Bug #41580: handle object class removal by simpleLDAP option logic

 * _ldap_addlist and _ldap_modlist no longer need to set object classes
 which are covered by options. These are automatically added/removed in
 simpleLDAP.
 
r70449 | Bug #41580: Replace old_samba_option / old_nagios_option

 * e.g. make it error prone if modify() is called multiple times
 
r70448 | Bug #41580: self.s4connector_present is already set in simpleLDAP

r70447 | Bug #41580: remove duplications covered by simpleLDAP

 * remove unnecessary constructors
 * mapping/(property)descriptions/options/alloc is set in simpleLDAP
 * self.save() is already called in the end of simpleLDAP.__init__()
 * self.ipRequest, self.oldPrimaryGroupDn, self.newPrimaryGroupDn is set
 in simpleComputer.__init__() / open()
 * self.default_dn is set in open()
 
r70446 | Bug #41580: evaluate the set of options from object classes

 * The set options of an instance are now "parsed" upon instanciation.
 Basically they are set from the object classes of the object.
 
 * In save() self.old_options must not set to the default options if the object doesn't exists.
 Therefore we also need to set self._exists = True after creating the object. And resetting it to False when removing the object.
 This was wrong previously and makes it (a little bit more) possible to further work with objects after some operations.
 
r70445 | Bug #41580: consider attribute name aliases
Comment 11 Florian Best univentionstaff 2016-06-27 14:11:23 CEST
(In reply to Jens Thorp-Hansen from comment #8)
> happened again: Ticket#2016062121000443
The problem here was that the modlist contained objectClass with 'inetorgperson' and 'inetOrgPerson'.
Comment 12 Florian Best univentionstaff 2016-06-27 17:43:16 CEST
r70641 | Bug #41580: normalize object class names to prevent errors in different case
→ Explicitly compare object classes case insensitive

r70640 | Bug #41580: fix storing of samba mungeddial (ctx flags) properties in settings/usertemplate
→ sambaMungeDial was not stored because the options defined 'samba' as required but in a user template no options are set.
Comment 13 Philipp Hahn univentionstaff 2016-06-28 08:14:00 CEST
<http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/AutotestJoin/lastCompletedBuild/SambaVersion=s3,Systemrolle=backup/testReport/00_checks/99check_log_files/test/>

SyntaxError: ('invalid syntax', ('/usr/lib/pymodules/python2.6/univention/admin/handlers/__init__.py', 827, 86, '\t\tmapping = {x.lower(): schema.get_obj(ldap.schema.models.ObjectClass, x).names[0] for x in ocs | unneeded_ocs | required_ocs}\n'))
[2016-06-27 18:16:33.591250] E: updater.log:1446, SyntaxError: ('invalid syntax', ('/usr/lib/pymodules/python2.6/univention/admin/handlers/
Comment 14 Philipp Hahn univentionstaff 2016-06-28 08:29:27 CEST
(In reply to Philipp Hahn from comment #13)
> <http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/
> AutotestJoin/lastCompletedBuild/SambaVersion=s3,Systemrolle=backup/
> testReport/00_checks/99check_log_files/test/>
> 
> SyntaxError: ('invalid syntax',
> ('/usr/lib/pymodules/python2.6/univention/admin/handlers/__init__.py', 827,
> 86, '\t\tmapping = {x.lower():
> schema.get_obj(ldap.schema.models.ObjectClass, x).names[0] for x in ocs |
> unneeded_ocs | required_ocs}\n'))
> [2016-06-27 18:16:33.591250] E: updater.log:1446, SyntaxError: ('invalid
> syntax', ('/usr/lib/pymodules/python2.6/univention/admin/handlers/

/usr/lib/pymodules/python2.6/univention/admin/handlers/__init__.py:827
@@ -827,1 +827,1 @@
- mapping = {x.lower(): schema.get_obj(ldap.schema.models.ObjectClass, x).names[0] for x in ocs | unneeded_ocs | required_ocs}
+ mapping = dict((x.lower(): schema.get_obj(ldap.schema.models.ObjectClass, x).names[0]) for x in ocs | unneeded_ocs | required_ocs)
Comment 15 Philipp Hahn univentionstaff 2016-06-28 09:23:04 CEST
(In reply to Philipp Hahn from comment #14)

/usr/lib/pymodules/python2.6/univention/admin/handlers/__init__.py:827
@@ -827,1 +827,1 @@
- mapping = {x.lower(): schema.get_obj(ldap.schema.models.ObjectClass, x).names[0] for x in ocs | unneeded_ocs | required_ocs}
+ mapping = dict((x.lower(), schema.get_obj(ldap.schema.models.ObjectClass, x).names[0]) for x in ocs | unneeded_ocs | required_ocs)

Also responsible for <http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/AutotestJoin/lastCompletedBuild/SambaVersion=s4,Systemrolle=backup/testReport/01_base/81alternativessl/test/>
Comment 16 Florian Best univentionstaff 2016-06-28 09:33:18 CEST
(In reply to Philipp Hahn from comment #15)
> (In reply to Philipp Hahn from comment #14)
> 
> /usr/lib/pymodules/python2.6/univention/admin/handlers/__init__.py:827
> @@ -827,1 +827,1 @@
> - mapping = {x.lower(): schema.get_obj(ldap.schema.models.ObjectClass,
> x).names[0] for x in ocs | unneeded_ocs | required_ocs}
> + mapping = dict((x.lower(), schema.get_obj(ldap.schema.models.ObjectClass,
> x).names[0]) for x in ocs | unneeded_ocs | required_ocs)
> 
> Also responsible for
> <http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/
> AutotestJoin/lastCompletedBuild/SambaVersion=s4,Systemrolle=backup/
> testReport/01_base/81alternativessl/test/>
Oh yes, thank you!
There was also the mistake that the 'case normalized object classes' were compared with the old ones. This is now done lowercase - and the case normalization is only done when changes in the object classes are necessary (so that the schema is only fetched then).

univention-directory-manager-modules (11.0.3-22):
r70665 | Bug #41580: fix python2.6 syntax
Comment 17 Florian Best univentionstaff 2016-06-29 15:28:56 CEST
univention-directory-manager-modules (11.0.3-24):
r70711 | Bug #41580: fix case insensitivity for objectclasses in extended attributes
Comment 18 Philipp Hahn univentionstaff 2016-06-30 14:30:29 CEST
FIXED: <http://jenkins.knut.univention.de:8080/job/UCS-4.1/job/UCS-4.1-2/job/AutotestJoin/lastCompletedBuild/SambaVersion=s3,Systemrolle=master/testReport/68_udm-extendedattribute/36_extended_attribute_removal_oc/test/>
 r70711

OK: r70445 r70446 r70447 r70448 r70449 r70450 r70451 r70452 r70453 r70468 r70469 r70503 r70514 r70585 r70599 r70639 r70640 r70641 r70656 r70665 r70711 r70728
OK:
 EA for users/user [option=PKI]
 Create user
 Add option PKI
 Set values
 Remove options PKI
 Check: OCs gone

OK:
  2 EAs for group/group
  Set/Delete one/both EAs

OK:
 2 EAs, 1 option=PKI
 Check: PKI enabled/disabled that EA
 Check: Other EA is not touched

OK: univention-directory-manager-modules.yaml
OK: errata-announce -V --only univention-directory-manager-modules.yaml

FYI: We should move <http://wiki.univention.de/index.php?title=Entwicklung_und_Integration_eigener_Module_in_Univention_Directory_Manager> into the developer guide, as it is UCS version dependent and now out-dated. Referenced by <http://docs.software-univention.de/developer-reference-4.1.html#udm:modules>
Comment 19 Janek Walkenhorst univentionstaff 2016-07-07 14:31:31 CEST
<http://errata.software-univention.de/ucs/4.1/208.html>
Comment 20 Florian Best univentionstaff 2016-07-26 16:21:17 CEST
*** Bug 21608 has been marked as a duplicate of this bug. ***