Univention Bugzilla – Bug 51290
UnicodeDecodeError: 'utf8' codec can't decode byte in position x : invalid start byte
Last modified: 2021-07-13 13:39:43 CEST
Version: 4.4-3 errata413 (Blumenthal) Internal server error during "appcenter/invoke_dry_run". Request: appcenter/invoke_dry_run Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 358, in __error_handling six.reraise(etype, exc, etraceback) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 261, in execute function.__func__(self, request, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/appcenter/__init__.py", line 398, in invoke_dry_run self.invoke(request) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/appcenter/__init__.py", line 127, in _deferred return func(self, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 639, in _decorated return function(self, request, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/decorators.py", line 181, in _response return function(self, request) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/appcenter/__init__.py", line 532, in invoke app = Apps().find(app_id) File "/usr/lib/python2.7/dist-packages/univention/appcenter/app_cache.py", line 84, in find apps = self.get_all_apps_with_id(app_id) File "/usr/lib/python2.7/dist-packages/univention/appcenter/app_cache.py", line 71, in get_all_apps_with_id for app in self.get_every_single_app(): File "/usr/lib/python2.7/dist-packages/univention/appcenter/app_cache.py", line 542, in get_every_single_app for app in app_cache.get_every_single_app(): File "/usr/lib/python2.7/dist-packages/univention/appcenter/app_cache.py", line 501, in get_every_single_app ret.extend(app_cache.get_every_single_app()) File "/usr/lib/python2.7/dist-packages/univention/appcenter/app_cache.py", line 362, in get_every_single_app if self._save_cache(): File "/usr/lib/python2.7/dist-packages/univention/appcenter/app_cache.py", line 223, in _save_cache dump([app.attrs_dict() for app in self._cache], fd, indent=2) File "/usr/lib/python2.7/json/__init__.py", line 189, in dump for chunk in iterable: File "/usr/lib/python2.7/json/encoder.py", line 431, in _iterencode for chunk in _iterencode_list(o, _current_indent_level): File "/usr/lib/python2.7/json/encoder.py", line 332, in _iterencode_list for chunk in chunks: File "/usr/lib/python2.7/json/encoder.py", line 390, in _iterencode_dict yield _encoder(value) UnicodeDecodeError: 'utf8' codec can't decode byte 0xb0 in position 8: invalid start byte Role: domaincontroller_master
reported again: Version: 4.4-5 errata703 (Blumenthal) Error: Internal server error during "setup/apps/query (wizard)". Request: setup/apps/query (wizard) Traceback (most recent call last): File "%PY2.7%/univention/management/console/base.py", line 359, in __error_handling six.reraise(etype, exc, etraceback) File "%PY2.7%/univention/management/console/base.py", line 262, in execute function.__func__(self, request, *args, **kwargs) File "%PY2.7%/univention/management/console/modules/decorators.py", line 321, in _response result = _multi_response(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 181, in _response return function(self, request) File "%PY2.7%/univention/management/console/modules/decorators.py", line 443, in _response return list(function(self, iterator, *nones)) File "%PY2.7%/univention/management/console/modules/decorators.py", line 289, in _fake_func yield function(self, *args) File "%PY2.7%/univention/management/console/modules/setup/__init__.py", line 759, in apps_query return util.get_apps(True) File "%PY2.7%/univention/management/console/modules/setup/util.py", line 792, in get_apps return [get.to_dict(app) for app in Apps().get_all_apps() if app.is_ucs_component()] File "%PY2.7%/univention/appcenter/app_cache.py", line 129, in get_all_apps for app in self.get_every_single_app(): File "%PY2.7%/univention/appcenter/app_cache.py", line 543, in get_every_single_app for app in app_cache.get_every_single_app(): File "%PY2.7%/univention/appcenter/app_cache.py", line 502, in get_every_single_app ret.extend(app_cache.get_every_single_app()) File "%PY2.7%/univention/appcenter/app_cache.py", line 363, in get_every_single_app if self._save_cache(): File "%PY2.7%/univention/appcenter/app_cache.py", line 224, in _save_cache dump([app.attrs_dict() for app in self._cache], fd, indent=2) File "/usr/lib/python2.7/json/__init__.py", line 189, in dump for chunk in iterable: File "/usr/lib/python2.7/json/encoder.py", line 431, in _iterencode for chunk in _iterencode_list(o, _current_indent_level): File "/usr/lib/python2.7/json/encoder.py", line 332, in _iterencode_list for chunk in chunks: File "/usr/lib/python2.7/json/encoder.py", line 390, in _iterencode_dict yield _encoder(value) UnicodeDecodeError: 'utf8' codec can't decode byte 0x8d in position 0: invalid start byte Role: None
Same problem here. Latest UCS 4.4-8 after trying to edit a user in UCS management console. Nothing obvious wrong in user settings. excerpt from ldap for specific user: dn: uid=Mustermann_U,ou=Benutzer,ou=Fertigung,ou=Abteilungen,ou=musterfirma,dc=musterfirma,dc=local uid: Mustermann_U krb5PrincipalName: Mustermann_U@musterfirma.LOCAL objectClass: krb5KDCEntry objectClass: person objectClass: automount objectClass: top objectClass: pkiUser objectClass: krb5Principal objectClass: organizationalPerson objectClass: univentionPWHistory objectClass: univentionMail objectClass: inetOrgPerson objectClass: shadowAccount objectClass: sambaSamAccount objectClass: posixAccount objectClass: univentionObject uidNumber: 2444 sambaAcctFlags: [U ] sambaPasswordHistory: I DELETED THIS FIELD sambaBadPasswordCount: 0 krb5MaxLife: 86400 cn: Uwe Mustermann mailForwardCopyToSelf: 0 krb5MaxRenew: 604800 gecos: Uwe Mustermann sambaBadPasswordTime: 0 userCertificate;binary:: I DELETED THIS FIELD - WAS BINARY description: RND loginShell: /bin/bash univentionObjectType: users/user krb5KDCFlags: 126 gidNumber: 5001 sambaPrimaryGroupSID: S-1-5-21-588273740-1646099605-1082013118-513 displayName: Uwe Mustermann mailPrimaryAddress: uwe.Mustermann@musterfirmamuc.tld sambaHomeDrive: H: l: Muenchen sambaSID: S-1-5-21-588273740-1646099605-1082013118-1117 sn: Mustermann pwhistory: I DELETED THIS FIELD homeDirectory: /home/Mustermann_U givenName: Uwe sambaNTPassword: I DELETED THIS FIELD krb5Key:: I DELETED THIS FIELD krb5Key:: I DELETED THIS FIELD krb5Key:: I DELETED THIS FIELD krb5Key:: I DELETED THIS FIELD krb5KeyVersionNumber: 11 userPassword:: I DELETED THIS FIELD shadowLastChange: 18778 sambaPwdLastSet: 1622439430 ------------- error message -------------------------------------- File "/usr/sbin/univention-management-console-module", line 120, in <module> notifier.loop() File "/usr/lib/python2.7/dist-packages/notifier/nf_generic.py", line 304, in loop step() File "/usr/lib/python2.7/dist-packages/notifier/nf_generic.py", line 295, in step __min_timer = dispatch.dispatcher_run() File "/usr/lib/python2.7/dist-packages/notifier/dispatch.py", line 75, in dispatcher_run if not disp(): File "/usr/lib/python2.7/dist-packages/notifier/threads.py", line 161, in _simple_threads_dispatcher task.announce() File "/usr/lib/python2.7/dist-packages/notifier/threads.py", line 139, in announce self._callback(self, self._result) File "/usr/lib/python2.7/dist-packages/notifier/__init__.py", line 104, in __call__ return self._function(*tmp, **self._kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 298, in thread_finished_callback self.finished(request.id, result) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 490, in finished self.result(res) File "/usr/lib/python2.7/dist-packages/univention/management/console/base.py", line 494, in result self.signal_emit('success', response) File "/usr/lib/python2.7/dist-packages/notifier/signals.py", line 84, in signal_emit self.__signals[signal].emit(*args) File "/usr/lib/python2.7/dist-packages/notifier/signals.py", line 49, in emit cb(*args) File "/usr/lib/python2.7/dist-packages/notifier/__init__.py", line 104, in __call__ return self._function(*tmp, **self._kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/modserver.py", line 141, in _reply self.response(msg) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/modserver.py", line 378, in response self.__queue += bytes(msg) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 140, in __str__ return self.__bytes__() File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 342, in __bytes__ return Message._formattedMessage(self._id, self._type, self.mimetype, self.command, body, self.arguments) File "/usr/lib/python2.7/dist-packages/univention/management/console/protocol/message.py", line 122, in _formattedMessage data = json.dumps(body) File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps return _default_encoder.encode(obj) File "/usr/lib/python2.7/json/encoder.py", line 207, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode return _iterencode(o, 0) UnicodeDecodeError: 'utf8' codec can't decode byte 0xfc in position 13: invalid start byte
Created attachment 10784 [details] debug patch Hello Stefan, can you apply this patch to the system, execute "systemctl restart univention-management-console-server", retry the action and paste the output here. patch -i /root/51290.patch -d / -p0
Thank you. Will send output via e-mail as it contains sensitive data.
(In reply to stefan.bauer from comment #4) > Thank you. > > Will send output via e-mail as it contains sensitive data. Thank you. The problem is the property 'certificateIssuerLocation' which has a value similar to 'F\x00l\x00o\x00r\x00i\x00\xe4\x00n' (I replaced the name with my name).
Thank you very much for your analysis. How can i modify this field from command-line? Is this a required field?
Created attachment 10785 [details] workaround patch (In reply to stefan.bauer from comment #6) > Thank you very much for your analysis. How can i modify this field from > command-line? Is this a required field? It is part of the user certificate. It's not required for a user object, but I don't know about your environment. You can either reupload a fixed new certificate: udm users/user modify --dn "$DN" --set userCertificate="$(base64 < userCertificate.der)" Or remove the currently set one: udm users/user modify --dn "$DN" --remove userCertificate Or you can upgrade to UCS 5.0 because it looks like this won't happen in UCS 5.0 (but I did not test it). Or you can temporarily patch your environment with the attached patch: patch -i /root/51290-2.patch -d / -p0
Awesome. Thank you a lot! That did the trick :)
for the record, we removed the certs from the affected users.