Bug 37402 - ucr.load() fails: 'NoneType' object has no attribute 'lower'
ucr.load() fails: 'NoneType' object has no attribute 'lower'
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UCR
UCS 4.0
Other Linux
: P5 normal (vote)
: UCS 4.2
Assigned To: Philipp Hahn
Florian Best
: interim-1
Depends on:
Blocks: 42912
  Show dependency treegraph
 
Reported: 2014-12-19 12:57 CET by Florian Best
Modified: 2017-04-04 18:29 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 3: Simply Wrong: The implementation doesn't match the docu
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.103
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2016102421000359, 2016102921000181, 2016110321000566, 2016110321000851, 2016110421000742, 2016111121000489
Bug group (optional): Error handling, External feedback
Max CVSS v3 score:
best: Patch_Available+


Attachments
Improve multi-threaded access in UCR (13.23 KB, patch)
2015-10-21 14:07 CEST, Philipp Hahn
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2014-12-19 12:57:31 CET
Traceback:
  File "/usr/lib/pymodules/python2.7/notifier/threads.py", line 82, in _run
    tmp = self._function()
  File "/usr/lib/pymodules/python2.7/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File
"/usr/lib/pymodules/python2.7/univention/management/console/modules/udm/__init__.py", line
507, in _thread
    ucr.load()
  File "/usr/lib/pymodules/python2.7/univention/config_registry/backend.py", line 106, in
load
    strict = self.is_true('ucr/encoding/strict')
  File "/usr/lib/pymodules/python2.7/univention/config_registry/backend.py", line 250, in
is_true
    value = self.get(key).lower()  # pylint: disable-msg=E1103

AttributeError: 'NoneType' object has no attribute 'lower'


Remark:
In Management-Console die DNS-Einstellungen überprüft

Version:
4.0-0 errata10 (Walle)
Comment 1 Philipp Hahn univentionstaff 2014-12-19 13:23:38 CET
The valueb must be set:
debian/univention-config.postinst:              univention-config-registry set 'ucr/encoding/strict?true'

Please check /etc/univention/base.conf for corruption!
Is univention-config fully configured: dpkg --configure univention-config
Comment 2 Florian Best univentionstaff 2015-05-05 14:55:53 CEST
Reported again, 4.0-1 errata162 (Walle)

Die Ausführung des Kommandos udm/query dns/dns ist fehlgeschlagen:

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/modules/__init__.py", line 176, in _decorated
    return function(self, request, *args, **kwargs)
  File "%PY2.7%/notifier/threads.py", line 82, in _run
    tmp = self._function()
  File "%PY2.7%/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "%PY2.7%/univention/management/console/modules/udm/__init__.py", line 520, in _thread
    ucr.load()
  File "%PY2.7%/univention/config_registry/backend.py", line 106, in load
    strict = self.is_true('ucr/encoding/strict')
  File "%PY2.7%/univention/config_registry/backend.py", line 250, in is_true
    value = self.get(key).lower()  # pylint: disable-msg=E1103
AttributeError: 'NoneType' object has no attribute 'lower'
Comment 3 Florian Best univentionstaff 2015-07-13 17:42:00 CEST
Reported again, 4.0-2 errata214
Comment 4 Florian Best univentionstaff 2015-08-31 08:33:07 CEST
Reported again, 4.0-3 errata292 (Walle)

(The same UMCP command)

Die Ausführung des Kommandos udm/query dns/dns ist fehlgeschlagen:

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 207, in _decorated
    return function(self, request, *args, **kwargs)
  File "%PY2.7%/notifier/threads.py", line 82, in _run
    tmp = self._function()
  File "%PY2.7%/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "%PY2.7%/univention/management/console/modules/udm/__init__.py", line 532, in _thread
    ucr.load()
  File "%PY2.7%/univention/config_registry/backend.py", line 106, in load
    strict = self.is_true('ucr/encoding/strict')
  File "%PY2.7%/univention/config_registry/backend.py", line 250, in is_true
    value = self.get(key).lower()  # pylint: disable-msg=E1103
AttributeError: 'NoneType' object has no attribute 'lower'
Comment 5 Florian Best univentionstaff 2015-10-19 11:17:25 CEST
Remark:
Access subnet via DHCP module

Version:
4.0-3 errata342 (Walle)

Execution of command 'udm/query dhcp/dhcp' has failed:

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 207, in _decorated
    return function(self, request, *args, **kwargs)
  File "%PY2.7%/notifier/threads.py", line 82, in _run
    tmp = self._function()
  File "%PY2.7%/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "%PY2.7%/univention/management/console/modules/udm/__init__.py", line 532, in _thread
    ucr.load()
  File "%PY2.7%/univention/config_registry/backend.py", line 106, in load
    strict = self.is_true('ucr/encoding/strict')
  File "%PY2.7%/univention/config_registry/backend.py", line 250, in is_true
    value = self.get(key).lower()  # pylint: disable-msg=E1103
AttributeError: 'NoneType' object has no attribute 'lower'
Comment 6 Philipp Hahn univentionstaff 2015-10-20 12:13:03 CEST
As the UCR "ucr/encoding/strict" is set very eraly, this looks like some serious base.conf corruption.
Comment 7 Florian Best univentionstaff 2015-10-20 12:46:05 CEST
But it is set in the postinst of univention-config. Maybe somethings goes wrong?
Shouldn't we do:
univention-config-registry set 'ucr/encoding/strict?true' || exit 1
?
Comment 8 Philipp Hahn univentionstaff 2015-10-21 14:07:13 CEST
Created attachment 7218 [details]
Improve multi-threaded access in UCR

I think is'a a multi-threading problem:
> File "%PY2.7%/univention/management/console/modules/udm/__init__.py", line 532, in _thread

UCR is *not* thread-safe;
I can trigger the bug with the attached test_backend_threading.py.

1. UMC uses a global shared instance of ConfigRegistry: univention.management.console.config.ucr

2. there is a (small) time window in load() between the dict.clear() and the loading, where UCR is empty or might miss the required values

The attached patch
- improves is_true() and is_false() to better handle None
- changes load() to not use dict.clear()
Comment 9 Florian Best univentionstaff 2016-04-04 10:11:33 CEST
Reported again, 3.2-8 errata410 (Borgfeld)
Comment 10 Florian Best univentionstaff 2016-05-20 06:42:34 CEST
Reported again, 4.1-2 errata183 (Vahr)

Remark: error after enabling DHCP boot policy
Comment 11 Florian Best univentionstaff 2016-08-15 15:11:51 CEST
Reported again, 4.1-2 errata220 (Vahr)
Comment 12 Florian Best univentionstaff 2016-09-13 12:53:23 CEST
Reported again, 4.1-3 errata262 (Vahr)
Comment 13 Philipp Hahn univentionstaff 2016-09-16 15:10:12 CEST
r72644 | Bug #37402 ucr: Make boolean handling more robust.
r72646 | Bug #37402 ucr: Improve multi-threaded load

Package: univention-config-registry
Version: 12.0.0-2A~4.2.0.201609161507
Branch: ucs_4.2-0
Comment 14 Philipp Hahn univentionstaff 2016-09-17 07:11:39 CEST
r72658 | Bug #37402 ucr: Fix check for corrupted base.conf
 Fix amd64 build

Package: univention-config-registry
Version: 12.0.0-3A~4.2.0.201609170709
Branch: ucs_4.2-0
Comment 15 Florian Best univentionstaff 2016-09-21 20:11:17 CEST
OK: Code review
OK: tests
OK: API-change, no functions use it like the following:
>>> ucr.is_true('update/secure_apt', None)
False
>>> ucr.is_true('update/secure_apt', None, 'true')
True
OK: Changelog
Comment 16 Florian Best univentionstaff 2016-09-28 20:46:25 CEST
The change revealed wrong UCR usage in Bug #42527 comment 3.
Comment 17 Florian Best univentionstaff 2016-10-25 16:48:46 CEST
Reported again, 4.1-2 errata202 (Vahr)
Comment 18 Florian Best univentionstaff 2016-11-09 17:05:11 CET
Reported again, Version: 4.1-3 errata318 (Vahr)
Comment 19 Florian Best univentionstaff 2016-11-09 17:05:30 CET
Reported again, 4.0-3 errata398 (Walle)
Comment 20 Florian Best univentionstaff 2016-11-09 17:05:49 CET
Reported again, 3.2-3 errata278 (Borgfeld)
Comment 21 Florian Best univentionstaff 2016-11-09 17:06:13 CET
Reported again, 4.1-3 errata324 (Vahr)
Comment 22 Stefan Gohmann univentionstaff 2017-04-04 18:29:02 CEST
UCS 4.2 has been released:
 https://docs.software-univention.de/release-notes-4.2-0-en.html
 https://docs.software-univention.de/release-notes-4.2-0-de.html

If this error occurs again, please use "Clone This Bug".