Univention Bugzilla – Bug 40882
TypeError: appcenter.app.AppAttribute is not JSON serializable
Last modified: 2016-03-17 14:28:55 CET
We received the following traceback, 4.1-1 errata128 (Vahr). I have no clue how that can happen. Die Ausführung des Kommandos appcenter/query ist fehlgeschlagen: Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 283, in execute function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 320, in _response self.finished(request.id, result[0]) File "%PY2.7%/univention/management/console/base.py", line 401, in finished self.result(res) File "%PY2.7%/univention/management/console/base.py", line 405, in result self.signal_emit('success', response) File "%PY2.7%/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "%PY2.7%/notifier/signals.py", line 41, in emit cb( *args ) File "%PY2.7%/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 118, in _reply self.response(msg) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 350, in response self.__queue += str(msg) File "%PY2.7%/univention/management/console/protocol/message.py", line 337, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "%PY2.7%/univention/management/console/protocol/message.py", line 130, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 321, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 237, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 311, in iterencode return _iterencode(o, 0) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 213, in default raise TypeError(repr(o) + " is not JSON serializable") TypeError: <univention.appcenter.app.AppAttribute object at 0x25d0c10> is not JSON serializable We should also adapt the __repr__ method of AppAttribute, … during fixing this bug.
Reported again
Probably hard to reproduce. It is a race condition: The preinst of univention-appcenter deletes the cache, but some other script rebuilds it _before the new python-univention-appcenter files are extracted_. Thus, newly introduced attributes are missing in the cache. But when the App Center is queried, the new attribute is in the python code, which leads to the error. The App object never overwrote license_description = AppAttribute() with self.license_description = None, thus the error message. Now, before reading the cache, app.py, meta.py, index.json.gz are os.stat'ed for last content change. If it is younger than the cache, the cache is deleted and rewritten with fresh data. Also, the cache file format has been changed from Pickle to JSON to force setting every attribute (at least with None). This change makes it also possible to compare attributes in the JSON file and the App class. But I did not include this for now, as the os.stat seems enough. univention-appcenter 5.0.20-22.133.201603140023
Reported again, 4.1-1 errata128 (Vahr)
Reproduce: ucr set 'appcenter/umc/update/always=false' rm /var/cache/univention-appcenter/.apps.* comment out website= and vendor= in /usr/share/pyshared/univention/appcenter/app.py call "univention-app list" uncomment the attributes in app.py open the appcenter in UMC
This isn't really safe against .py/.pyc changes: >>> from univention.appcenter import AppManager >>> AppManager._relevant_master_files() set(['/var/cache/univention-appcenter/.index.json.gz', '/usr/lib/pymodules/python2.7/univention/appcenter/app.py', '/usr/lib/pymodules/python2.7/univention/appcenter/meta.pyc']) Isn't there a better thing than os.stat()ing - e.g. versionize the API? A simple touch .apps.de.json .apps.en.json (or modify the ini files remote and remove some attributes + univention-app list) causes the bug to reappear - in theory it could trigger null-pointer exceptions especially for required attributes - the harmless case is that the attributes are just not shown or shown as 'null'.
Image for the os.stat scenario: E.g. somewhere in postinst (after the new files app.py are placed with mtime=1 in the file system) a currently still running process (univention-app update, appcenter-UMC with the old code) ends and writes the cache file (mtime=2). → Broken - now more harmless - situation again. The cache file is also nowhere removed anymore.
The described scenarios are possible, but improbable. But to fix this once and for all, one would need some kind of API version and put it into the cache. I have implemented a far more simple check in univention-appcenter 5.0.20-25.136.201603151120 It compares the attribute _names_ in the cache with those in the App class. If they differ, the cache is rewritten. This can still fail to identify an outdated cache: - web_interface_proxy_scheme = AppAttribute(default='both', choices=['http', 'https', 'both']) + web_interface_proxy_scheme = AppAttribute(default='http', choices=['http', 'https', 'both']) would not be recognized. I think it is good enough the way it now, though.
Reported again (twice), 4.1-1 errata128 (Vahr)
OK: This is nice now. OK: YAML
<http://errata.software-univention.de/ucs/4.1/130.html>