Univention Bugzilla – Bug 40485
AppCenter: wrong encoding causes crash
Last modified: 2021-12-17 11:14:25 CET
Re received the following traceback, 4.1-0 errata14 (Vahr). Execution of command 'appcenter/query' has failed: Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 283, in execute function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 320, in _response self.finished(request.id, result[0]) File "%PY2.7%/univention/management/console/base.py", line 401, in finished self.result(res) File "%PY2.7%/univention/management/console/base.py", line 405, in result self.signal_emit('success', response) File "%PY2.7%/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "%PY2.7%/notifier/signals.py", line 41, in emit cb( *args ) File "%PY2.7%/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 118, in _reply self.response(msg) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 350, in response self.__queue += str(msg) File "%PY2.7%/univention/management/console/protocol/message.py", line 337, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "%PY2.7%/univention/management/console/protocol/message.py", line 130, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 321, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 237, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 311, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xae in position 9: invalid start byte It seems that there was a ® which was not encoded in UTF-8 (\xc2\xae) but in latin-1 (\xae) which causes this error.
Probably best to prevent this on our App Center server, not ignoring it (or guessing the encoding) on the UCS server side.
We now have tools on the App Center server to prevent this.
(In reply to Dirk Wiesenthal from comment #2) > We now have tools on the App Center server to prevent this. But it happened again. Version: 4.2-2 errata159 (Lesum) Remark: Not able to install via UCS - Will not fetch docker image.... Owncloud installs, but I did not want OC, but I suppose I will have to.
Reported again: Version: 4.2-3 errata305 (Lesum)
Reported again: Version: 4.3-0 errata83 (Neustadt) Interner Server-Fehler in "appcenter/packages/query". Request: appcenter/packages/query Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 253, in execute function.__func__(self, request, *args, **kwargs) File "%PY2.7%/univention/management/console/modules/decorators.py", line 192, in _response return function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 320, in _response self.finished(request.id, result[0]) File "%PY2.7%/univention/management/console/base.py", line 434, in finished self.result(res) File "%PY2.7%/univention/management/console/base.py", line 438, in result self.signal_emit('success', response) File "%PY2.7%/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "%PY2.7%/notifier/signals.py", line 41, in emit cb( *args ) File "%PY2.7%/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 124, in _reply self.response(msg) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 354, in response self.__queue += str(msg) File "%PY2.7%/univention/management/console/protocol/message.py", line 308, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "%PY2.7%/univention/management/console/protocol/message.py", line 114, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 380, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 291, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 373, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xfc in position 7: invalid start byte
Reported again: Version: 4.3-0 errata35 (Neustadt) Same Traceback as Comment #5
Version: 4.3-0 errata112 (Neustadt) Internal server error during "appcenter/docker/progress". Request: appcenter/docker/progress Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 253, in execute function.__func__(self, request, *args, **kwargs) File "%PY2.7%/univention/management/console/modules/decorators.py", line 320, in _response self.finished(request.id, result[0]) File "%PY2.7%/univention/management/console/base.py", line 434, in finished self.result(res) File "%PY2.7%/univention/management/console/base.py", line 438, in result self.signal_emit('success', response) File "%PY2.7%/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "%PY2.7%/notifier/signals.py", line 41, in emit cb( *args ) File "%PY2.7%/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 124, in _reply self.response(msg) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 354, in response self.__queue += str(msg) File "%PY2.7%/univention/management/console/protocol/message.py", line 308, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "%PY2.7%/univention/management/console/protocol/message.py", line 114, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 380, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 291, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 373, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xe2 in position 13: invalid continuation byte
Reported again: Version: 4.3-2 errata229 (Neustadt) Interner Server-Fehler in "setup/apps/query (wizard)". Request: setup/apps/query (wizard) Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 253, in execute function.__func__(self, request, *args, **kwargs) File "%PY2.7%/univention/management/console/modules/decorators.py", line 318, in _response result = _multi_response(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 192, in _response return function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 440, in _response return list(function(self, iterator, *nones)) File "%PY2.7%/univention/management/console/modules/decorators.py", line 286, in _fake_func yield function(self, *args) File "%PY2.7%/univention/management/console/modules/setup/__init__.py", line 749, in apps_query return util.get_apps(True) File "%PY2.7%/univention/management/console/modules/setup/util.py", line 790, in get_apps return [get.to_dict(app) for app in Apps().get_all_apps() if app.is_ucs_component()] File "%PY2.7%/univention/appcenter/app_cache.py", line 119, in get_all_apps for app in self.get_every_single_app(): File "%PY2.7%/univention/appcenter/app_cache.py", line 533, in get_every_single_app for app in app_cache.get_every_single_app(): File "%PY2.7%/univention/appcenter/app_cache.py", line 492, in get_every_single_app ret.extend(app_cache.get_every_single_app()) File "%PY2.7%/univention/appcenter/app_cache.py", line 353, in get_every_single_app if self._save_cache(): File "%PY2.7%/univention/appcenter/app_cache.py", line 214, in _save_cache dump([app.attrs_dict() for app in self._cache], fd, indent=2) File "/usr/lib/python2.7/json/__init__.py", line 189, in dump for chunk in iterable: File "/usr/lib/python2.7/json/encoder.py", line 431, in _iterencode for chunk in _iterencode_list(o, _current_indent_level): File "/usr/lib/python2.7/json/encoder.py", line 332, in _iterencode_list for chunk in chunks: File "/usr/lib/python2.7/json/encoder.py", line 390, in _iterencode_dict yield _encoder(value) UnicodeDecodeError: 'utf8' codec can't decode byte 0xc3 in position 4: invalid continuation byte
This issue has been filled against UCS 4.1. The maintenance with bug and security fixes for UCS 4.1 has ended on 5st of April 2018. Customers still on UCS 4.1 are encouraged to update to UCS 4.3. Please contact your partner or Univention for any questions. If this issue still occurs in newer UCS versions, please use "Clone this bug" or simply reopen the issue. In this case please provide detailed information on how this issue is affecting you.
Reported again: Version: 4.4-0 errata59 (Blumenthal) Interner Server-Fehler in "appcenter/packages/query". Request: appcenter/packages/query Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 253, in execute function.__func__(self, request, *args, **kwargs) File "%PY2.7%/univention/management/console/modules/decorators.py", line 192, in _response return function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 320, in _response self.finished(request.id, result[0]) File "%PY2.7%/univention/management/console/base.py", line 462, in finished self.result(res) File "%PY2.7%/univention/management/console/base.py", line 466, in result self.signal_emit('success', response) File "%PY2.7%/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "%PY2.7%/notifier/signals.py", line 41, in emit cb( *args ) File "%PY2.7%/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 128, in _reply self.response(msg) File "%PY2.7%/univention/management/console/protocol/modserver.py", line 359, in response self.__queue += str(msg) File "%PY2.7%/univention/management/console/protocol/message.py", line 304, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "%PY2.7%/univention/management/console/protocol/message.py", line 110, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/json/encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xfc in position 7: invalid start byte
Reported again: 4.4-3 errata385 (Blumenthal) Internal server error during "appcenter/docker/progress". Request: appcenter/docker/progress Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 358, in __error_handling six.reraise(etype, exc, etraceback) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 261, in execute function.__func__(self, request, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 323, in _response self.finished(request.id, result[0]) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 478, in finished self.result(res) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 482, in result self.signal_emit('success', response) File "%PY2.7%/notifier/signals.py", line 75, in signal_emit self.__signals[ signal ].emit( *args ) File "%PY2.7%/notifier/signals.py", line 41, in emit cb( *args ) File "%PY2.7%/notifier/__init__.py", line 104, in __call__ return self._function( *tmp, **self._kwargs ) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/modserver.py", line 130, in _reply self.response(msg) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/modserver.py", line 361, in response self.__queue += str(msg) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 305, in __str__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 111, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/json/encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xa9 in position 13: invalid start byte Role: domaincontroller_master
Version: 5.0-1 errata175 Error: Internal server error during "appcenter/packages/query". Request: appcenter/packages/query Traceback (most recent call last): File "%PY3%/univention/management/console/base.py", line 344, in __error_handling six.reraise(etype, exc, etraceback) File "%PY3%/six.py", line 693, in reraise raise value File "%PY3%/univention/management/console/base.py", line 247, in execute function.__func__(self, request, *args, **kwargs) File "%PY3%/univention/management/console/modules/decorators.py", line 181, in _response return function(self, request) File "%PY3%/univention/management/console/modules/decorators.py", line 321, in _response result = _multi_response(self, request) File "%PY3%/univention/management/console/modules/decorators.py", line 181, in _response return function(self, request) File "%PY3%/univention/management/console/modules/decorators.py", line 443, in _response return list(function(self, iterator, *nones)) File "%PY3%/univention/management/console/modules/decorators.py", line 289, in _fake_func yield function(self, *args) File "%PY3%/univention/management/console/modules/appcenter/__init__.py", line 789, in packages_query result.append(self._package_to_dict(package, full=False)) File "%PY3%/univention/management/console/modules/appcenter/__init__.py", line 906, in _package_to_dict 'summary': candidate.summary, File "%PY3%/apt/package.py", line 606, in summary return records.short_desc if records is not None else None UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 7: invalid start byte Role: domaincontroller_master