First Last Prev Next    This bug is not in your last search results.
Bug 49775 - Interner Server-Fehler in "modules": UnicodeDecodeError: 'utf8' codec can't decode byte 0xa4 in position 0: invalid start byte / unexpected end of data
Interner Server-Fehler in "modules": UnicodeDecodeError: 'utf8' codec can't d...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 4.4-0-errata
Assigned To: Florian Best
Dirk Wiesenthal
:
: 49825 (view as bug list)
Depends on:
Blocks:
  Show dependency tree / graph
 
Reported: 2019-07-02 11:55 CEST by Florian Best
Modified: 2024-02-23 13:34 CET (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.400
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support: Yes
Flags outvoted (downgraded) after PO Review:
Ticket number: 2019070221000285, 2019070221000249, 2019070121000303, 2019070321000318, 2019070421000307, 2019071021000322, 2019070921000932, 2019070821000121, 2019070221000285
Bug group (optional): External feedback, Regression
Max CVSS v3 score:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2019-07-02 11:55:18 CEST 
Version: 4.4-0 errata168 (Blumenthal)

Remark: die installation von letsencrypt ist mit fehlermeldung abgebrochen ..

Traceback(c7e748692a5f8b44f4d2c705e3d0d369):
Interner Server-Fehler in "modules".
Request: modules

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 260, in execute
    function.__func__(self, request, *args, **kwargs)
  File "%PY2.7%/univention/management/console/protocol/session.py", line 270, in handle_request_get
    self.finished(msg.id, method(msg))
  File "%PY2.7%/univention/management/console/protocol/session.py", line 319, in handle_request_get_modules
    'name': self.i18n._(flavor.name, translationId),
  File "%PY2.7%/univention/management/console/locales.py", line 182, in _
    self[domain] = I18N(self.locale, domain)
  File "%PY2.7%/univention/management/console/locales.py", line 79, in __init__
    self.load(locale, domain)
  File "%PY2.7%/univention/management/console/locales.py", line 109, in load
    self.mofile = polib.mofile(filename)
  File "/usr/lib/python2.7/dist-packages/polib.py", line 171, in mofile
    return _pofile_or_mofile(mofile, 'mofile', **kwargs)
  File "/usr/lib/python2.7/dist-packages/polib.py", line 86, in _pofile_or_mofile
    instance = parser.parse()
  File "/usr/lib/python2.7/dist-packages/polib.py", line 1714, in parse
    enumerate(msgstr.split(b('\0'))))
  File "/usr/lib/python2.7/dist-packages/polib.py", line 1740, in _build_entry
    msgstr_plural[k] = msgstr_plural[k].decode(encoding)
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xec in position 0: unexpected end of data

Role: domaincontroller_master

Version: 4.4-0 errata168 (Blumenthal)

Remark: nix geht mehr

Traceback(c7e748692a5f8b44f4d2c705e3d0d369):
Interner Server-Fehler in "modules".
Request: modules

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 260, in execute
    function.__func__(self, request, *args, **kwargs)
  File "%PY2.7%/univention/management/console/protocol/session.py", line 270, in handle_request_get
    self.finished(msg.id, method(msg))
  File "%PY2.7%/univention/management/console/protocol/session.py", line 319, in handle_request_get_modules
    'name': self.i18n._(flavor.name, translationId),
  File "%PY2.7%/univention/management/console/locales.py", line 182, in _
    self[domain] = I18N(self.locale, domain)
  File "%PY2.7%/univention/management/console/locales.py", line 79, in __init__
    self.load(locale, domain)
  File "%PY2.7%/univention/management/console/locales.py", line 109, in load
    self.mofile = polib.mofile(filename)
  File "/usr/lib/python2.7/dist-packages/polib.py", line 171, in mofile
    return _pofile_or_mofile(mofile, 'mofile', **kwargs)
  File "/usr/lib/python2.7/dist-packages/polib.py", line 86, in _pofile_or_mofile
    instance = parser.parse()
  File "/usr/lib/python2.7/dist-packages/polib.py", line 1717, in parse
    entry = self._build_entry(msgid=msgid, msgstr=msgstr)
  File "/usr/lib/python2.7/dist-packages/polib.py", line 1735, in _build_entry
    kwargs['msgstr'] = msgstr.decode(encoding)
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xa4 in position 0: invalid start byte

Role: domaincontroller_master
Comment 1 Florian Best univentionstaff 2019-07-02 11:56:01 CEST 
There seem to be some app which ships an broken .mo translation file containing ISO8859-1 instead of UTF-8.
Comment 2 Florian Best univentionstaff 2019-07-03 08:40:20 CEST 
Reported again, 4.4-0 errata168 (Blumenthal) - UCS@school 4.4 v2
Comment 3 Florian Best univentionstaff 2019-07-04 09:30:06 CEST 
Reported again, 4.4-0 errata168 (Blumenthal)
Comment 4 Florian Best univentionstaff 2019-07-04 09:31:08 CEST 
It must be a module using flavors. So maybe something which extends UDM.
Comment 5 Dirk Wiesenthal univentionstaff 2019-07-10 18:15:36 CEST 
Has something to do with the list of installed apps. This list causes the exception:

admindiary-backend=1.0
admindiary-frontend=1.0
cups=2.2.1
dhcp-server=12.0
kopano-core=8.7.1.0
kopano-webapp=3.5.5.2276
kopano-webmeetings=3.0.1.100
kvm=2.8
letsencrypt=1.2.2-8
nagios=4.3
openvpn4ucs=1.1.16
samba4=4.10
self-service=4.0
uvmm=7.0
z-push-kopano=2.4.5
4.3/collabora=4.0.3.1
4.3/onlyoffice-ds=5.3.0.243
4.1/nextcloud=15.0.7-0
Comment 6 Dirk Wiesenthal univentionstaff 2019-07-10 18:19:00 CEST 
The Apps UMC module creates its flavors on the fly. The translations are created within a UCR template:

/etc/univention/templates/files/usr/share/univention-management-console/i18n/de/apps.mo

This template builds up the translations and prints po.to_binary() at the end.

Since http://errata.software-univention.de/ucs/4.4/167.html we use universtal_newlines=True in the subprocess when generating the final file with python tokens.
Comment 7 Dirk Wiesenthal univentionstaff 2019-07-10 18:21:57 CEST 
To reproduce:

Edit /etc/univention/templates/files/usr/share/univention-management-console/i18n/de/apps.mo

Do not use "apps = Apps().get_all_locally_installed_apps()" in line 24, instead, do this:

apps = []
wanted = "admindiary-backend=1.0 admindiary-frontend=1.0 cups=2.2.1 dhcp-server=12.0 kopano-core=8.7.1.0 kopano-webapp=3.5.5.2276 kopano-webmeetings=3.0.1.100 kvm=2.8 letsencrypt=1.2.2-8 nagios=4.3 openvpn4ucs=1.1.16 samba4=4.10 self-service=4.0 uvmm=7.0 z-push-kopano=2.4.5 4.3/collabora=4.0.3.1 4.3/onlyoffice-ds=5.3.0.243 4.1/nextcloud=15.0.7-0"
for app_string in wanted.split():
  app_id, app_version = app_string.split('=')
  if '/' in app_id:
    app_id = app_id.split('/')[-1]
  app = Apps().find(app_id, app_version=app_version)
  apps.append(app)


ucr commit /usr/share/univention-management-console/i18n/de/apps.mo

python -c 'import polib; polib.mofile("/usr/share/univention-management-console/i18n/de/apps.mo")'
Comment 8 Dirk Wiesenthal univentionstaff 2019-07-10 18:23:08 CEST 
*** Bug 49825 has been marked as a duplicate of this bug. ***
Comment 11 Christian Völker univentionstaff 2019-07-11 11:49:14 CEST 
Flagging "Waiting Support" as for customer seems to be no workaround available and UMC is not useable at all.
Comment 12 Florian Best univentionstaff 2019-07-11 14:14:22 CEST 
UCR has been fixed/reverted, so that it behaves like prior erratum 167.
The commit message contains an explanation and a suggestion for a further patch for python3.

The error handling in the UMC-Server has been enhanced to ignore broken translation files. And error message is logged in that case.

univention-config-registry (14.0.0-10)
de7dd0a42cde | Bug #49775: fix UCR templates printing binary data

univention-management-console.yaml
2ee35068f5b4 | YAML Bug #49775
3485bca52fec | Bug #49775: error handling when loading translation files has been improoved

univention-management-console (11.0.4-26)
3485bca52fec | Bug #49775: error handling when loading translation files has been improoved

univention-config-registry.yaml
2ee35068f5b4 | YAML Bug #49775


Merged to UCS 4.4-1:
univention-config-registry (14.0.0-10)
2a77ca862038 | Bug #49775: fix UCR templates printing binary data

univention-management-console (11.0.4-26)
4a97e4ecbc02 | Bug #49775: error handling when loading translation files has been improoved
Comment 13 Dirk Wiesenthal univentionstaff 2019-07-11 15:06:52 CEST 
UMC does not bail out on corrupt .mo files: OK
UCR can write binary data: OK
YAML: OK
Merge to 4.4-1: OK
First Last Prev Next    This bug is not in your last search results.