Bug 33101 - UCR does not fullfill Python dict contract
UCR does not fullfill Python dict contract
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UCR
UCS 4.2
All Linux
: P5 enhancement with 4 votes (vote)
: UCS 4.2
Assigned To: Philipp Hahn
Janek Walkenhorst
: interim-3
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-04 13:30 CET by Philipp Hahn
Modified: 2017-04-04 18:29 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Cleanup, Error handling, Troubleshooting
Max CVSS v3 score:
hahn: Patch_Available+


Attachments
Use collection.MutableMapping instead of dict as base class (4.07 KB, patch)
2016-12-01 11:31 CET, Philipp Hahn
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Hahn univentionstaff 2013-11-04 13:30:58 CET
univention.config_registry.ConfigRegistry inherits from dict() and overwrites several methods, which breaks the Python contract for dictionaries <http://docs.python.org/2.6/library/stdtypes.html#mapping-types-dict>:

from univention.config_registry import ConfigRegistry
ucr = ConfigRegistry()
ucr.load()
dict(ucr) # → {}
list(ucr.itervalues()) # → []
list(ucr.iteritems()) # → []

<http://docs.python.org/2.6/library/userdict.html#UserDict.DictMixin> or <http://docs.python.org/2.6/library/collections.html#MutableMapping> should be considered
Comment 1 Stefan Gohmann univentionstaff 2013-11-21 08:04:47 CET
To get an idea about the priority. For what do we need it?
Comment 2 Philipp Hahn univentionstaff 2013-11-21 17:33:37 CET
(In reply to Stefan Gohmann from comment #1)
> To get an idea about the priority. For what do we need it?

Additional time to debug unexpected behavior and find a work-around.

I know of at least two occurrences where people used ucr.iteritems() first and later on had to change the code to use ucr.items() adding a comment warning about ucr not following the dict contract. This is no serious bug, but still costs time to debug and you have to know, that ucr is special ...
Comment 3 Stefan Gohmann univentionstaff 2013-11-21 20:30:56 CET
(In reply to Philipp Hahn from comment #2)
> (In reply to Stefan Gohmann from comment #1)
> > To get an idea about the priority. For what do we need it?
> 
> Additional time to debug unexpected behavior and find a work-around.
> 
> I know of at least two occurrences where people used ucr.iteritems() first
> and later on had to change the code to use ucr.items() adding a comment
> warning about ucr not following the dict contract. This is no serious bug,
> but still costs time to debug and you have to know, that ucr is special ...

OK, as long as we did not change it we could add a short hint at the dev guide:
 http://docs.univention.de/developer-reference-3.2.html#ucr:usage:python

Everybody who starts with the UCR programming can read it.
Comment 4 Stefan Gohmann univentionstaff 2013-11-29 07:10:25 CET
Re-check for UCS 4.
Comment 5 Florian Best univentionstaff 2016-10-21 15:17:54 CEST
Still miserable in UCS 4.2.
Comment 6 Philipp Hahn univentionstaff 2016-10-21 15:45:34 CEST
Again: r73453

(In reply to Stefan Gohmann from comment #3)
> OK, as long as we did not change it we could add a short hint at the dev
> guide:
>  http://docs.univention.de/developer-reference-3.2.html#ucr:usage:python
> 
> Everybody who starts with the UCR programming can read it.

and will forget it after 1 week of doing other things.
Comment 7 Florian Best univentionstaff 2016-11-04 18:12:22 CET
Occurred again in svn r74123.
Comment 9 Philipp Hahn univentionstaff 2016-12-01 11:31:17 CET
Created attachment 8266 [details]
Use collection.MutableMapping instead of dict as base class
Comment 10 Eduard Mai univentionstaff 2017-02-09 18:20:26 CET
Debugged again while trying to use it with str.format() and key based formatting. Statements like the following also access the (unused) dict structure created by inheriting from dict instead of the ones in ConfigRegistry._registry:

'{version/version}'.format(ucr)

This results in KeyError.
Comment 11 Philipp Hahn univentionstaff 2017-02-10 14:38:56 CET
r76603 | Bug #33101 ucr: Fix UCR dict contract

Package: univention-config-registry
Version: 12.0.1-3A~4.2.0.201702101358
Branch: ucs_4.2-0

(In reply to Eduard Mai from comment #10)
> Debugged again while trying to use it with str.format() and key based
> formatting. Statements like the following also access the (unused) dict
> structure created by inheriting from dict instead of the ones in
> ConfigRegistry._registry:
> 
> '{version/version}'.format(ucr)
> 
> This results in KeyError.

Your "syntax" is wrong here, see last two lines:

print '{version/version}'.format({'version/version':42})
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# KeyError: 'version/version'

from univention.config_registry import ConfigRegistry
ucr = ConfigRegisry()
ucr.load()
print ucr['version/version']
print ucr.get('version/version')
print '%(version/version)s' % ucr
print '{version/version}'.format(**ucr)
print '{0[version/version]}'.format(ucr)
Comment 12 Florian Best univentionstaff 2017-02-10 14:44:03 CET
1. Please comment something about this API change:

Prior:
>>> ucr['foo']
''
>>> ucr.get('foo')
''
>>> ucr.items(getscope=ConfigRegistry.SCHEDULE)
…

Afterwards:
>>> ucr['foo']
>>> ucr.get('foo')

2. 
$ ucr search foo
E: your request could not be fulfilled
try `univention-config-registry --help` for more information
>>> from univention.config_registry.frontend import handler_search
>>> handler_search([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pymodules/python2.7/univention/config_registry/frontend.py", line 329, in handler_search
    for key, (scope, value) in ucr.items(getscope=True):
TypeError: items() got an unexpected keyword argument 'getscope'
>>> ucr.items(getscope=ConfigRegistry.SCHEDULE)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: items() got an unexpected keyword argument 'getscope'
Comment 13 Philipp Hahn univentionstaff 2017-02-10 16:27:01 CET
(In reply to Florian Best from comment #12)
> 1. Please comment something about this API change:
> Prior:
> >>> ucr['foo'] is ''
...
> Afterwards:
> >>> ucr['foo'] is None

This was caused by "foo" being set to '' in your test:
> # grep ^foo: /etc/univention/base*.conf
> /etc/univention/base.conf:foo: 


> 2. 
> >>> from univention.config_registry.frontend import handler_search;handler_search([])
...
> TypeError: items() got an unexpected keyword argument 'getscope'

r76606 | Bug #33101 ucr: Re-add item(getscope=True) custom implementation
 Fixed

Package: univention-config-registry
Version: 12.0.1-4A~4.2.0.201702101623
Branch: ucs_4.2-0
Comment 14 Janek Walkenhorst univentionstaff 2017-03-20 15:51:09 CET
OK
Changelog: OK
Comment 15 Stefan Gohmann univentionstaff 2017-04-04 18:29:26 CEST
UCS 4.2 has been released:
 https://docs.software-univention.de/release-notes-4.2-0-en.html
 https://docs.software-univention.de/release-notes-4.2-0-de.html

If this error occurs again, please use "Clone This Bug".