Bug 45936 - LC_ALL=C lsb_release -si due to encoding
LC_ALL=C lsb_release -si due to encoding
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UCR
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3
Assigned To: Felix Botner
Philipp Hahn
: interim-2
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-12-21 13:57 CET by Felix Botner
Modified: 2018-03-14 14:38 CET (History)
2 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
updater.log (1.82 MB, text/plain)
2017-12-27 19:32 CET, Stefan Gohmann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Botner univentionstaff 2017-12-21 13:57:17 CET
this breaks the jenkins tests

during the update a package postinst fails with

Traceback (most recent call last):
  File "/usr/bin/lsb_release", line 95, in <module>
    main()
  File "/usr/bin/lsb_release", line 59, in main
    distinfo = lsb_release.get_distro_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 343, in get_distro_information
    lsbinfo = get_lsb_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 323, in get_lsb_information
    for line in lsb_release_file:
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 264: ordinal not in range(128)

in 4.3 the following command fails,

->  LC_ALL=C lsb_release -si
Traceback (most recent call last):
  File "/usr/bin/lsb_release", line 95, in <module>
    main()
  File "/usr/bin/lsb_release", line 59, in main
    distinfo = lsb_release.get_distro_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 343, in get_distro_information
    lsbinfo = get_lsb_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 323, in get_lsb_information
    for line in lsb_release_file:
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 264: ordinal not in range(128)

i guess the umlauts in our ucr warning are the problem
Comment 1 Florian Best univentionstaff 2017-12-21 14:03:27 CET
(In reply to Felix Botner from comment #0)
> i guess the umlauts in our ucr warning are the problem
Yes. The file is opened with ANSI_X3.4-1968 encoding:
<_io.TextIOWrapper name='/etc/lsb-release' mode='r' encoding='ANSI_X3.4-1968'>
Comment 2 Felix Botner univentionstaff 2017-12-21 14:12:42 CET
ü -> ue in WARNING_TEXT
univention-config-registry
Comment 3 Florian Best univentionstaff 2017-12-21 14:20:18 CET
(In reply to Felix Botner from comment #2)
> ü -> ue in WARNING_TEXT
> univention-config-registry

I think the approach is wrong. Why don't you just change it in this way?:

diff --git a/base/univention-base-files/conffiles/etc/lsb-release b/base/univention-base-files/conffiles/etc/lsb-release
index 486d2df..29ea75d 100644
--- a/base/univention-base-files/conffiles/etc/lsb-release
+++ b/base/univention-base-files/conffiles/etc/lsb-release
@@ -1,4 +1,4 @@
-@%@UCRWARNING=# @%@
+@%@UCRWARNINGASCII=# @%@
 
 DISTRIB_ID=Univention
 DISTRIB_RELEASE="@%@version/version@%@-@%@version/patchlevel@%@ errata@%@version/erratalevel@%@"
Comment 4 Florian Best univentionstaff 2017-12-21 14:22:12 CET
I meant @%@UCRWARNING_ASCII=# @%@
Comment 5 Felix Botner univentionstaff 2017-12-21 14:56:16 CET
(In reply to Florian Best from comment #3)
> (In reply to Felix Botner from comment #2)
> > ü -> ue in WARNING_TEXT
> > univention-config-registry
> 
> I think the approach is wrong. Why don't you just change it in this way?:
> 
> diff --git a/base/univention-base-files/conffiles/etc/lsb-release
> b/base/univention-base-files/conffiles/etc/lsb-release
> index 486d2df..29ea75d 100644
> --- a/base/univention-base-files/conffiles/etc/lsb-release
> +++ b/base/univention-base-files/conffiles/etc/lsb-release
> @@ -1,4 +1,4 @@
> -@%@UCRWARNING=# @%@
> +@%@UCRWARNINGASCII=# @%@
>  
>  DISTRIB_ID=Univention
>  DISTRIB_RELEASE="@%@version/version@%@-@%@version/patchlevel@%@
> errata@%@version/erratalevel@%@"

great, so we have extra functions just to allow 'ü' in the warning text, seems a bit excessive to me ...

changed UCRWARNING to UCRWARNINGASCII and then to the correct UCRWARNING_ASCII (in conffiles/etc/lsb-release in univention-base-files)

but i did not revert the ü -> ue change, sue me
Comment 6 Stefan Gohmann univentionstaff 2017-12-27 19:31:31 CET
The upgrade still fails. I guess univention-base-files is not upgraded at that point.
Comment 7 Stefan Gohmann univentionstaff 2017-12-27 19:32:05 CET
Created attachment 9329 [details]
updater.log
Comment 8 Stefan Gohmann univentionstaff 2017-12-27 20:09:43 CET
Please undo the following svn commit once this issue is fixed. And please add a changelog:

r17917:
Temporarily disabled the lsb_release call in slapd postinst for milestone 1 (Bug #45936)

Modified Paths
    openldap/4.3-0-0-ucs/2.4.45+dfsg-1~bpo9+1/85_disable_move_and_dump.patch
Comment 9 Felix Botner univentionstaff 2018-01-09 15:42:41 CET
(In reply to Stefan Gohmann from comment #8)
> Please undo the following svn commit once this issue is fixed. And please
> add a changelog:
> 
> r17917:
> Temporarily disabled the lsb_release call in slapd postinst for milestone 1
> (Bug #45936)
> 
> Modified Paths
>     openldap/4.3-0-0-ucs/2.4.45+dfsg-1~bpo9+1/85_disable_move_and_dump.patch

replaced the lsb_release test in 85_disable_move_and_dump.patch" "with a test for /usr/sbin/ucr, in case ucr is found, distributor="Univention" is assumed

Package: openldap
Version: 2.4.45+dfsg-1~bpo9+1A~4.3.0.201801091316
Branch: ucs_4.3-0
Comment 10 Philipp Hahn univentionstaff 2018-01-11 15:06:08 CET
fddb1cfe3fee | Bug #45936: use UCRWARNINGASCII in lsb-release
bb5b2c9e31e9 | Bug #45936: only ucr ascii in warning text
7c3ad870a5b4 | Bug #45936: use UCRWARNING_ASCII in lsb-release
r17948 | Bug #45936: replace lsb_release test with ucr
r17917 | Temporarily disabled the lsb_release call in slapd postinst for milestone 1 (Bug #45936)

OK: LC_ALL=C lsb_release -si
OK: slapd

FAIL: No changelog
RFA: Please revert the change to univention-base-files. Having non-ASCII is a good test case
Comment 11 Philipp Hahn univentionstaff 2018-01-15 09:21:59 CET
04305d3c61 Bug #45936 test: Add LSB test
Comment 12 Felix Botner univentionstaff 2018-01-15 12:59:53 CET
(In reply to Philipp Hahn from comment #10)
> fddb1cfe3fee | Bug #45936: use UCRWARNINGASCII in lsb-release
> bb5b2c9e31e9 | Bug #45936: only ucr ascii in warning text
> 7c3ad870a5b4 | Bug #45936: use UCRWARNING_ASCII in lsb-release
> r17948 | Bug #45936: replace lsb_release test with ucr
> r17917 | Temporarily disabled the lsb_release call in slapd postinst for
> milestone 1 (Bug #45936)
> 
> OK: LC_ALL=C lsb_release -si
> OK: slapd
> 
> FAIL: No changelog
added

> RFA: Please revert the change to univention-base-files. Having non-ASCII is
> a good test case

why?, if we revert back we will get an error running

-> LC_ALL=C lsb_release -i

Traceback (most recent call last):
  File "/usr/bin/lsb_release", line 95, in <module>
    main()
  File "/usr/bin/lsb_release", line 59, in main
    distinfo = lsb_release.get_distro_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 343, in get_distro_information
    lsbinfo = get_lsb_information()
  File "/usr/lib/python3/dist-packages/lsb_release.py", line 323, in get_lsb_information
    for line in lsb_release_file:
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 264: ordinal not in range(128)

So i think the change is still necessary, even though the patched slapd package no longer uses this, but maybe others ...
Comment 13 Philipp Hahn univentionstaff 2018-01-15 16:02:32 CET
(In reply to Felix Botner from comment #12)
> (In reply to Philipp Hahn from comment #10)
> > fddb1cfe3fee | Bug #45936: use UCRWARNINGASCII in lsb-release
> > bb5b2c9e31e9 | Bug #45936: only ucr ascii in warning text
> > 7c3ad870a5b4 | Bug #45936: use UCRWARNING_ASCII in lsb-release
> > r17948 | Bug #45936: replace lsb_release test with ucr
> > r17917 | Temporarily disabled the lsb_release call in slapd postinst for
> > milestone 1 (Bug #45936)
> > 
> > OK: LC_ALL=C lsb_release -si
> > OK: slapd
> > 
> > FAIL: No changelog
> added

OK: ff02828b46 Bug #45936: info

> > RFA: Please revert the change to univention-base-files. Having non-ASCII is
> > a good test case
> 
> why?, if we revert back we will get an error running

Sorry, I ment
- revert bb5b2c9e31e9cd529c41cc1ccb942d6b0f17cbe7 from univention-config to again output 'ü' 
- keep 7c3ad870a5b4c2c599f63a8559e52d6b22b2550f for univention-base-filregistryes to use UCRWARNING_ASCII
Comment 14 Felix Botner univentionstaff 2018-01-17 11:16:47 CET
> 
> Sorry, I ment
> - revert bb5b2c9e31e9cd529c41cc1ccb942d6b0f17cbe7 from univention-config to
> again output 'ü' 
> - keep 7c3ad870a5b4c2c599f63a8559e52d6b22b2550f for
> univention-base-filregistryes to use UCRWARNING_ASCII

hmm, why should we change this back?

That 'ü' caused an error in lsb_release and we now have a test for that, but how do we know that there are no other problems?

I would prefer to play it save and have only asci in the ucr warning.
Comment 15 Philipp Hahn univentionstaff 2018-01-17 11:33:44 CET
(In reply to Felix Botner from comment #14)
> > Sorry, I ment
> > - revert bb5b2c9e31e9cd529c41cc1ccb942d6b0f17cbe7 from univention-config to
> > again output 'ü' 
> > - keep 7c3ad870a5b4c2c599f63a8559e52d6b22b2550f for
> > univention-base-filregistryes to use UCRWARNING_ASCII
> 
> hmm, why should we change this back?
> 
> That 'ü' caused an error in lsb_release and we now have a test for that, but
> how do we know that there are no other problems?
> 
> I would prefer to play it save and have only asci in the ucr warning.

Because we have UCRWARNING_ASCII for those cases.
Cause the word is spelled with an 'ü'.
Because you have not added a changelog entry for UCR (and UBF).
Comment 16 Stefan Gohmann univentionstaff 2018-03-14 14:38:53 CET
UCS 4.3 has been released:
 https://docs.software-univention.de/release-notes-4.3-0-en.html
 https://docs.software-univention.de/release-notes-4.3-0-de.html

If this error occurs again, please use "Clone This Bug".