Univention Bugzilla – Bug 40882
TypeError: appcenter.app.AppAttribute is not JSON serializable
Last modified: 2016-03-17 14:28:55 CET
We received the following traceback, 4.1-1 errata128 (Vahr).
I have no clue how that can happen.
Die Ausführung des Kommandos appcenter/query ist fehlgeschlagen:
Traceback (most recent call last):
File "%PY2.7%/univention/management/console/base.py", line 283, in execute
File "%PY2.7%/univention/management/console/modules/decorators.py", line 320, in _response
File "%PY2.7%/univention/management/console/base.py", line 401, in finished
File "%PY2.7%/univention/management/console/base.py", line 405, in result
File "%PY2.7%/notifier/signals.py", line 75, in signal_emit
self.__signals[ signal ].emit( *args )
File "%PY2.7%/notifier/signals.py", line 41, in emit
cb( *args )
File "%PY2.7%/notifier/__init__.py", line 104, in __call__
return self._function( *tmp, **self._kwargs )
File "%PY2.7%/univention/management/console/protocol/modserver.py", line 118, in _reply
File "%PY2.7%/univention/management/console/protocol/modserver.py", line 350, in response
self.__queue += str(msg)
File "%PY2.7%/univention/management/console/protocol/message.py", line 337, in __str__
return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments)
File "%PY2.7%/univention/management/console/protocol/message.py", line 130, in _formattedMessage
data = json.dumps(body)
File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 321, in dumps
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 237, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 311, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 213, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <univention.appcenter.app.AppAttribute object at 0x25d0c10> is not JSON serializable
We should also adapt the __repr__ method of AppAttribute, … during fixing this bug.
Probably hard to reproduce. It is a race condition: The preinst of univention-appcenter deletes the cache, but some other script rebuilds it _before the new python-univention-appcenter files are extracted_. Thus, newly introduced attributes are missing in the cache. But when the App Center is queried, the new attribute is in the python code, which leads to the error.
The App object never overwrote
license_description = AppAttribute()
with self.license_description = None, thus the error message.
Now, before reading the cache, app.py, meta.py, index.json.gz are os.stat'ed for last content change. If it is younger than the cache, the cache is deleted and rewritten with fresh data.
Also, the cache file format has been changed from Pickle to JSON to force setting every attribute (at least with None).
This change makes it also possible to compare attributes in the JSON file and the App class. But I did not include this for now, as the os.stat seems enough.
Reported again, 4.1-1 errata128 (Vahr)
ucr set 'appcenter/umc/update/always=false'
comment out website= and vendor= in /usr/share/pyshared/univention/appcenter/app.py
call "univention-app list"
uncomment the attributes in app.py
open the appcenter in UMC
This isn't really safe against .py/.pyc changes:
>>> from univention.appcenter import AppManager
set(['/var/cache/univention-appcenter/.index.json.gz', '/usr/lib/pymodules/python2.7/univention/appcenter/app.py', '/usr/lib/pymodules/python2.7/univention/appcenter/meta.pyc'])
Isn't there a better thing than os.stat()ing - e.g. versionize the API?
A simple touch .apps.de.json .apps.en.json (or modify the ini files remote and remove some attributes + univention-app list) causes the bug to reappear - in theory it could trigger null-pointer exceptions especially for required attributes - the harmless case is that the attributes are just not shown or shown as 'null'.
Image for the os.stat scenario:
E.g. somewhere in postinst (after the new files app.py are placed with mtime=1 in the file system) a currently still running process (univention-app update, appcenter-UMC with the old code) ends and writes the cache file (mtime=2).
→ Broken - now more harmless - situation again.
The cache file is also nowhere removed anymore.
The described scenarios are possible, but improbable. But to fix this once and for all, one would need some kind of API version and put it into the cache.
I have implemented a far more simple check in
It compares the attribute _names_ in the cache with those in the App class. If they differ, the cache is rewritten. This can still fail to identify an outdated cache:
- web_interface_proxy_scheme = AppAttribute(default='both', choices=['http', 'https', 'both'])
+ web_interface_proxy_scheme = AppAttribute(default='http', choices=['http', 'https', 'both'])
would not be recognized. I think it is good enough the way it now, though.
Reported again (twice), 4.1-1 errata128 (Vahr)
OK: This is nice now.