Bug 42404 - univention.admin.mapping.mapValue('foo', '') does not raise KeyError
univention.admin.mapping.mapValue('foo', '') does not raise KeyError
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UDM (Generic)
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.1-4-errata
Assigned To: Florian Best
Philipp Hahn
:
Depends on:
Blocks: 41986
  Show dependency treegraph
 
Reported: 2016-09-15 13:18 CEST by Florian Best
Modified: 2017-01-05 11:22 CET (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.103
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Cleanup, Error handling, Troubleshooting
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2016-09-15 13:18:13 CEST
Inconsistent behavior in the mapValue() method of univention.admin.mapping.mapValue causes Bugs like Bug #41986.

>>> m.mapValue('foo', '')        
''
>>> m.mapValue('foo', '1')
KeyError: 'foo'

mapName('foo') doesn't raise KeyError at all.
A greater problem is that mapRewrite() & co. are used for different purposes (e.g. creating filters for search, map the value for the add/modlist).
Comment 1 Florian Best univentionstaff 2016-11-10 16:49:27 CET
mapValue is used in multiple UDM handlers but only with keys which are registered in the mapping.
mapValue is used in mapDiffAl which is nowhere used.
mapValue is used in mapDiff which is used to create the simpleLdap._ldap_modlist() with the values got from simpleLdap.diff(). These might contain properties which are not present in the mapping. But as they are not in the mapping they have to be added in _ldap_modlist. The KeyError there is already catched.
mapValue is called in mapRewrite which is called in a lot of lookup functions. And this is the actual case where this error happens:

def mapRewrite(filter, mapping):
»   try:
»   »   k = mapping.mapName(filter.variable)
»   »   v = mapping.mapValue(filter.variable, filter.value)
»   except KeyError:
»   »   return
»   filter.variable = k 
»   filter.value = v 

mapName(foobar) → return an empty string
if mapValue(foobar, '') also returns an empty string the filter is broken: '(=)'

univention-directory-manager-modules (11.0.3-44):
r74294 | Bug #42404: fix mapping.mapValue to raise KeyError

univention-directory-manager-modules.yaml:
r74296 | YAML Bug #42404

Merge to UCS 4.2:
univention-directory-manager-modules (12.0.5-3):
r74295 | Bug #42404: fix mapping.mapValue to raise KeyError
Comment 2 Florian Best univentionstaff 2016-11-10 17:04:07 CET
By the way: This bug is the reason for all the reported tracebacks we got in the past until we appended '*' to the search value in Bug #24341 / Bug #16387 svn r35873.
Comment 3 Florian Best univentionstaff 2016-11-10 17:24:42 CET
Since we have Bug #42388 this is now also again reproducible:

ucr set directory/manager/web/auto_substring_search=false directory/manager/web/allow_wildcard_search=false

$ python
>>> import univention.admin.modules
>>> univention.admin.modules.update()
>>> for module in univention.admin.modules.modules.values():
...  if not hasattr(module, 'mapping') # users/self users/password
...   continue
...  for name, property in module.property_descriptions.items():
...   if not module.mapping.mapName(name) and not property.dontsearch:
...    print module.module, name
→ This reveals a quite long list. Picking one example:

umc-client -s 10.200.27.117 -U Administrator -P univention COMMAND udm/query -f dns/dns -e -o '{"container":"all","objectType":"dns/reverse_zone","objectProperty":"contact","objectPropertyValue":""}'

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/notifier/threads.py", line 82, in _run
    tmp = self._function()
  File "/usr/lib/pymodules/python2.7/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/udm/__init__.py", line 543, in _thread
    result = module.search(container, objectProperty, objectPropertyValue, superordinate, scope=scope, hidden=hidden)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/udm/udm_ldap.py", line 88, in _decorated
    return method(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/ldap.py", line 141, in _decorated
    result = func(*args, **kwargs)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/udm/udm_ldap.py", line 476, in search
    result = self.module.lookup(None, ldap_connection, filter_s, base=container, superordinate=superordinate, scope=scope, sizelimit=sizelimit)
  File "/usr/lib/pymodules/python2.7/univention/admin/handlers/dns/reverse_zone.py", line 302, in lookup
    for dn, attrs in lo.search(unicode(filter), base, scope, [], unique, required, timeout, sizelimit):
  File "/usr/lib/pymodules/python2.7/univention/admin/uldap.py", line 339, in search
    raise univention.admin.uexceptions.ldapError('%s: %s' % (_err2str(msg), filter))
ldapError: Bad search filter: (&(objectClass=dNSZone)(relativeDomainName=@)(|(zoneName=*.in-addr.arpa)(zoneName=*.ip6.arpa))(=))
Comment 4 Philipp Hahn univentionstaff 2016-12-06 17:32:02 CET
OK: UCS-4.1-4 r74477 r74296 r74294 r75046 + r75048
 mapping: New style python class
 unregister: Clever use of .pop()
 mapName: Clever use of .get()
 unmapName: Clever use of .get()
 mapValue: Previously returned '' in case of empty value, now may raise KeyError first if map_name is unknown. This is what is described in comment 0.
 unmapValue: Refactored
 mapRewrite: Previously returned empty LDAP attribute name if unknown UDM property name was given
OK: UCS-4.2-0 r75047 r75049 r74295

OK: univention-directory-manager-modules.yaml
OK: errata-announce -V --only univention-directory-manager-modules.yaml

OK: aptitude install '?source-package(univention-directory-manager-modules)~i' # 11.0.3-51.1445.201612061711
OK: zless /usr/share/doc/python-univention-directory-manager/changelog.Debian.gz
OK: umc-client -s localhost -U Administrator -P univention COMMAND udm/query -f dns/dns -e -o '{"container":"all","objectType":"dns/reverse_zone","objectProperty":"contact","objectPropertyValue":""}'
Comment 5 Janek Walkenhorst univentionstaff 2017-01-05 11:22:39 CET
<http://errata.software-univention.de/ucs/4.1/367.html>