Univention Bugzilla – Bug 52194
locale.setlocale() used wrong
Last modified: 2021-05-25 16:01:50 CEST
locale.setlocale() is used wrong in may places: $ git grep -n setlocale > base/univention-lib/python/admember.py:895: locale.setlocale(locale.LC_TIME, (None, None)) # 'C' as env['LC_ALL'] some lines earlier > base/univention-lib/python/admember.py:900: locale.setlocale(locale.LC_TIME, old_locale) locate is a process-wide setting and affects other thread. A library should never use setlocale() See <https://docs.python.org/3/library/locale.html#background-details-hints-tips-and-caveats> > base/univention-lib/python/atjobs.py:233: locale.setlocale(locale.LC_TIME, 'C') > base/univention-lib/python/atjobs.py:246: locale.setlocale(locale.LC_TIME, timeLocale) dito > management/univention-management-console-module-adtakeover/umc/python/adtakeover/takeover.py:835: locale.setlocale(locale.LC_TIME, (None, None)) # 'C' as env['LC_ALL'] some lines earlier > management/univention-management-console-module-adtakeover/umc/python/adtakeover/takeover.py:840: locale.setlocale(locale.LC_TIME, old_locale) dito > base/univention-system-setup/umc/python/setup/__init__.py:112: _locale.setlocale(_locale.LC_ALL, str(self.locale)) > base/univention-system-setup/umc/python/setup/__init__.py:685: _locale.setlocale(_locale.LC_ALL, str(locale)) > base/univention-system-setup/umc/python/setup/__init__.py:688: _locale.setlocale(_locale.LC_ALL, 'C') Switcheing the locale is expensive and should be avoided. > management/univention-appcenter/apps/python/apps/__init__.py:52: locale.setlocale(locale.LC_ALL, str(self.locale)) > management/univention-appcenter/umc/python/appcenter/__init__.py:163: locale.setlocale(locale.LC_ALL, str(self.locale)) > management/univention-directory-manager-rest/src/univention/admin/rest/__main__.py:71: locale.setlocale(locale.LC_MESSAGES, language) > packaging/univention-directory-manager-module-example/scripts/ip-phone-tool:46:univention.admin.localization.locale.setlocale(univention.admin.localization.locale.LC_ALL, '') > virtualization/univention-virtual-machine-manager-daemon/scripts-dev/uvmm-memleak:17:from locale import setlocale, LC_ALL > virtualization/univention-virtual-machine-manager-daemon/scripts-dev/uvmm-memleak:39: setlocale(LC_ALL, '') > virtualization/univention-virtual-machine-manager-daemon/src/univention-virtual-machine-manager:1021: locale.setlocale(locale.LC_ALL, '') > virtualization/univention-virtual-machine-manager-daemon/src/univention-virtual-machine-manager-daemon:194: locale.setlocale(locale.LC_ALL, '') okay. > base/univention-system-setup/umc/python/setup/setup_script.py:50: locale.setlocale(locale.LC_ALL, locale.getdefaultlocale()) > management/univention-appcenter/scripts/univention-app:41: locale.setlocale(locale.LC_ALL, locale.getdefaultlocale()) > management/univention-directory-reports/univention-directory-reports:66: locale.setlocale(locale.LC_ALL, locale.getdefaultlocale()) This breaks the default "C" locale, as getdefaultlocale() returns "en_US" instead of "C", which might be unavailable (for example inside the build environment!) Just use the empty sting which will do the right thing: locale.setlocale(locale.LC_ALL, "") > management/univention-management-console/scripts/univention-management-console-client:287: locale.setlocale(locale.LC_ALL, os.environ['LC_ALL']) 'LC_ALL' is only the most preferred setting, LC_CTYPE, LANG and LANGUAGE should also be considered. See <https://docs.python.org/3/library/locale.html#locale.getdefaultlocale> Just use the empty sting which will do the right thing: locale.setlocale(locale.LC_ALL, "") > management/univention-management-console/scripts/univention-management-console-module:95: locale.setlocale(locale.LC_MESSAGES, str(locale_obj)) > management/univention-management-console/scripts/univention-management-console-server:124: locale.setlocale(locale.LC_MESSAGES, locale.normalize(self.options.language)) > management/univention-management-console/scripts/univention-management-console-server:125: locale.setlocale(locale.LC_CTYPE, locale.normalize(self.options.language)) > management/univention-management-console/src/univention/management/console/base.py:194: locale.setlocale(locale.LC_MESSAGES, _locale) > management/univention-management-console/src/univention/management/console/base.py:195: locale.setlocale(locale.LC_CTYPE, _locale) probably okay > management/univention-management-console-module-udm/umc/python/udm/__init__.py:186: locale.setlocale(locale.LC_TIME, _locale) don't know
https://git.knut.univention.de/univention/ucs/-/tree/phahn/52194-locale
[5.0-0] 6ba967ee76 Bug #52194: Fix locale.getdefaultlocate() usage base/univention-system-setup/umc/python/setup/setup_script.py | 5 ++--- management/univention-appcenter/scripts/univention-app | 2 +- management/univention-directory-reports/univention-directory-reports | 4 ++-- management/univention-management-console/scripts/univention-management-console-client | 4 ++-- 4 files changed, 7 insertions(+), 8 deletions(-) [5.0-0] 2d90576002 Bug #52194 umcd: Add/fox PEP 484 type annotations .../src/univention/management/console/auth.py | 19 ++++++----- .../src/univention/management/console/pam.py | 49 ++++++++++++++++------------- .../src/univention/management/console/protocol/client.py | 37 +++++++++++++++++----- .../src/univention/management/console/protocol/message.py | 24 +++++++++----- .../src/univention/management/console/protocol/session.py | 14 +++++++-- 5 files changed, 98 insertions(+), 45 deletions(-) [5.0-0] 1224a7842f Bug #52194 umc-cmd: Add PEP 484 type annotaions .../scripts/univention-management-console-client | 34 +++++++++++++++++++---------- 1 file changed, 22 insertions(+), 12 deletions(-) [5.0-0] 120f51ca21 Bug #52194 umc-cmd: Split argument parsing into sub .../scripts/univention-management-console-client | 76 ++++++++++++++++------------- 1 file changed, 41 insertions(+), 35 deletions(-) [5.0-0] 89618dbb90 Bug #52194 umc-cmd: import datetime from datetime management/univention-management-console/scripts/univention-management-console-client | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) [5.0-0] d5c371c552 Bug #52194 umc-cmd: Simplify timing .../scripts/univention-management-console-client | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) [5.0-0] 663d25e551 Bug #52194 lib/i18n: Fix setting locale base/univention-lib/python/i18n.py | 62 ++++++++++++-------- base/univention-lib/unittests/test_i18n.py | 155 ++++++++++++++++++++++++------------------------- 2 files changed, 112 insertions(+), 105 deletions(-) [5.0-0] edfb3ed84d Bug #52194: Fix locale.getdefaultlocale() usage doc/changelog/changelog-5.0-0.xml | 3 +++ 1 file changed, 3 insertions(+) For 4.4-7 only the first commit has been cherry-picked: This is required for building the packages on GitLab using Docker, as there no locale package is installed an `en_US.UTF-8` is unavailable: [4.4-7] ce4ab61c56 Bug #52194: Fix locale.getdefaultlocate() usage base/univention-system-setup/umc/python/setup/setup_script.py | 5 ++--- management/univention-appcenter/scripts/univention-app | 2 +- management/univention-directory-reports/univention-directory-reports | 4 ++-- management/univention-management-console/scripts/univention-management-console-client | 4 ++-- 4 files changed, 7 insertions(+), 8 deletions(-) The packages have not been rebuilt in UCS 4.4-7!
Cherry-picked also to 4.4-7 on @fbest request: [4.4-7] 64b54c1f4f Bug #52194 umcd: Add/fox PEP 484 type annotations .../src/univention/management/console/auth.py | 19 ++++++----- .../src/univention/management/console/pam.py | 49 ++++++++++++++++------------- .../src/univention/management/console/protocol/client.py | 37 +++++++++++++++++----- .../src/univention/management/console/protocol/message.py | 24 +++++++++----- .../src/univention/management/console/protocol/session.py | 14 +++++++-- 5 files changed, 98 insertions(+), 45 deletions(-) [4.4-7] 30d000d3c4 Bug #52194 umc-cmd: Add PEP 484 type annotaions .../scripts/univention-management-console-client | 34 +++++++++++++++++++---------- 1 file changed, 22 insertions(+), 12 deletions(-) [4.4-7] d2c91d5675 Bug #52194 umc-cmd: Split argument parsing into sub .../scripts/univention-management-console-client | 76 ++++++++++++++++------------- 1 file changed, 41 insertions(+), 35 deletions(-) [4.4-7] 45bc80584d Bug #52194 umc-cmd: import datetime from datetime management/univention-management-console/scripts/univention-management-console-client | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) [4.4-7] 3c03721756 Bug #52194 umc-cmd: Simplify timing .../scripts/univention-management-console-client | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-)
git:663d25e551bae915c9e9a2df08d3b3ec6176c6d8 broke the System Setup. The problem with Bug #51633 was that we didn't validate a correct locale prior. Now we are doing this. So we have to be backwards compatible for e.g. System Setup which uses ":UTF-8" in the locale. I am okay fixing this manually in USS an do the API change now in UCS 5. It should be the only place where it fails. 02.12.20 11:35:08.465 MODULE ( WARN ) : Exception during saving the settings: File "/usr/lib/python2.7/dist-packages/notifier/threads.py", line 80, in _run result = self._function() File "/usr/lib/python2.7/dist-packages/notifier/__init__.py", line 105, in __call__ return self._function(*tmp, **self._kwargs) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/setup/__init__.py", line 277, in _thread util.auto_complete_values_for_join(values) File "/usr/lib/python2.7/dist-packages/univention/management/console/modules/setup/util.py", line 238, in auto_complete_values_for_join current_locale = Locale(ucr.get('locale/default', 'en_US.UTF-8:UTF-8')) File "/usr/lib/python2.7/dist-packages/univention/lib/i18n.py", line 77, in __init__ self.parse(locale) File "/usr/lib/python2.7/dist-packages/univention/lib/i18n.py", line 100, in parse raise I18N_Error('attribute does not match locale specification language[_territory][.codeset][@modifier]') I18N_Error: attribute does not match locale specification language[_territory][.codeset][@modifier]
I fixed this in: univention-system-setup (13.0.2-5) f1f265fbe59e | Bug #52194: Bug #51633: pass valid locale to univention.i18n.Locale
(In reply to Florian Best from comment #4) > git:663d25e551bae915c9e9a2df08d3b3ec6176c6d8 broke the System Setup. Yes, I reverted the change from Bug #51633 because the change there was wrong. > The problem with Bug #51633 was that we didn't validate a correct locale > prior. Now we are doing this. So we have to be backwards compatible for e.g. > System Setup which uses ":UTF-8" in the locale. > I am okay fixing this manually in USS an do the API change now in UCS 5. It > should be the only place where it fails. That information would have helped at Bug #51633: There it only claimed "something was wrong" without giving evidence; your traceback would have helped. (In reply to Florian Best from comment #5) > I fixed this in: > > univention-system-setup (13.0.2-5) > f1f265fbe59e | Bug #52194: Bug #51633: pass valid locale to > univention.i18n.Locale OK.
OK: univention.lib.i18n.Locale OK: setlocale() in USS, App-Center, UMC, UDR OK: UMC (mypy & co) OK: partly UCS 4.4-7 backport without rebuild OK: changelog entry
Sorry for hijacking this bug :( The `umc-command` program is broken: https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-7/job/AutotestJoin/SambaVersion=s4,Systemrolle=master/lastCompletedBuild/testReport/59_udm/21_request_new_license/master091/ eval_options has been renamed to eval_option. And there is one negation to much in the argument parser. Since it is just a small change I already checked it in. I hope that's ok. [4.4-7 621fd3ad33] fixup! Bug #52194 umc-cmd: Split argument parsing into sub [5.0-0 90c6fa693e] fixup! Bug #52194 umc-cmd: Split argument parsing into sub
(In reply to Jürn Brodersen from comment #8) > Sorry for hijacking this bug :( > > The `umc-command` program is broken: > https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-7/job/ > AutotestJoin/SambaVersion=s4,Systemrolle=master/lastCompletedBuild/ > testReport/59_udm/21_request_new_license/master091/ > > eval_options has been renamed to eval_option. > And there is one negation to much in the argument parser. > > Since it is just a small change I already checked it in. I hope that's ok. > > > [4.4-7 621fd3ad33] fixup! Bug #52194 umc-cmd: Split argument parsing into sub > [5.0-0 90c6fa693e] fixup! Bug #52194 umc-cmd: Split argument parsing into sub Oh, thanks very much!
UCS 5.0 has been released: https://docs.software-univention.de/release-notes-5.0-0-en.html https://docs.software-univention.de/release-notes-5.0-0-de.html If this error occurs again, please use "Clone This Bug".