Bug 40874 - AppCenter is unable to load if syntax error in license_types.ini
AppCenter is unable to load if syntax error in license_types.ini
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - App-Center
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1-1-errata
Assigned To: Dirk Wiesenthal
Florian Best
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-10 06:40 CET by Florian Best
Modified: 2016-03-17 14:28 CET (History)
1 user (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Error handling, External feedback
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2016-03-10 06:40:58 CET
http://appcenter.software-univention.de/meta-inf/license_types.ini contained:
[freemium]
Description=Free commercial use and some functions or services are liable to costs
Description[de]=Kostenlose kommerzielle Nutzung und einige Funktionen oder Dienste sind
kostenpflichtig

If we don't prevent that the file can be corrupted we should handle the error. Otherwise one is unable to use the appcenter at all. If the license-types cannot be load it should be just left out.

We received this traceback twice tonight:
Execution of command 'appcenter/query' has failed:
Execution of command 'apps/get' has failed:
Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 283, in execute
    function(self, request)
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 318, in _response
    result = _multi_response(self, request)
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 462, in _response
    return list(function(self, iterator, *nones))
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 284, in _fake_func
    yield function(self, *args)
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 191, in query
    return domain.to_dict(apps)
  File "%PY2.7%/univention/appcenter/actions/domain.py", line 106, in to_dict
    app_dict = get.to_dict(app)
  File "%PY2.7%/univention/appcenter/actions/docker_get.py", line 45, in to_dict
    ret = super(Get, cls).to_dict(app)
  File "%PY2.7%/univention/appcenter/actions/get.py", line 112, in to_dict
    ret['is_ucs_component'] = app.is_ucs_component()
  File "%PY2.7%/univention/appcenter/app.py", line 619, in is_ucs_component
    app = App.from_ini(self.get_ini_file(), locale=False)
  File "%PY2.7%/univention/appcenter/app.py", line 532, in from_ini
    value = _get_license_descriptions().get(license_description_section)
  File "%PY2.7%/univention/appcenter/app.py", line 101, in _get_license_descriptions
    license_parser = _read_ini_file(os.path.join(CACHE_DIR, '.license_types.ini'))
  File "%PY2.7%/univention/appcenter/app.py", line 67, in _read_ini_file
    parser.readfp(f)
  File "/usr/lib/python2.7/ConfigParser.py", line 324, in readfp
    self._read(fp, filename)
  File "/usr/lib/python2.7/ConfigParser.py", line 546, in _read
    raise e
ParsingError: File contains parsing errors: /var/cache/univention-appcenter/.license_types.ini
	[line 15]: 'kostenpflichtig\n'
Comment 1 Dirk Wiesenthal univentionstaff 2016-03-14 00:32:34 CET
Using more general ParsingError instead of MissingSectionHeaderError in
  univention-appcenter 5.0.20-22.133.201603140023

Now these files are ignored (actually they are treated as emtpy). Holds for every ini file (license_types, categories, ratings, but also every App ini).

This would result in untranslated categories and so on, but not in tracebacks. Apps are not shown if the ini is malformed (because the Parser is emtpy and ID= is missing).
Comment 2 Florian Best univentionstaff 2016-03-14 08:59:16 CET
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/management/console/base.py", line 283, in execute
    function(self, request)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 318, in _response
    result = _multi_response(self, request)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 462, in _response
    return list(function(self, iterator, *nones))
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/decorators.py", line 284, in _fake_func
    yield function(self, *args)
  File "/usr/lib/pymodules/python2.7/univention/management/console/modules/appcenter/__init__.py", line 191, in query
    return domain.to_dict(apps)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/domain.py", line 106, in to_dict
    app_dict = get.to_dict(app)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_get.py", line 45, in to_dict
    ret = super(Get, cls).to_dict(app)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/get.py", line 112, in to_dict
    ret['is_ucs_component'] = app.is_ucs_component()
  File "/usr/lib/pymodules/python2.7/univention/appcenter/app.py", line 621, in is_ucs_component
    self._is_ucs_component = 'UCS Components' in app.categories
AttributeError: 'NoneType' object has no attribute 'categories'
Comment 3 Dirk Wiesenthal univentionstaff 2016-03-14 11:07:43 CET
There is a inconistent behaviour in RawConfigParser. Working around it now. Also include categories.ini to use _read_ini_file.

Fixed an error along the lines which I thought was the cause of the traceback.
Comment 4 Dirk Wiesenthal univentionstaff 2016-03-14 13:53:17 CET
Fixed one more bug that happened when after dev scripts, no license_types.ini were found on the server. ("local app center")
Comment 5 Florian Best univentionstaff 2016-03-15 07:14:28 CET
license_types: OK, malformed license-types is ignored
categories: OK, malformed categories.ini causes the categories not to be translated
ratings: OK, malformed ratings.ini causes the ratings to not being displayed
App ini: OK, malformed app.ini causes the app to not be present.

REOPEN: there is no error message in the logfiles about broken license-types, categories, ratings. Even the app.ini logfile message is really short and only a WARNing.
Comment 6 Dirk Wiesenthal univentionstaff 2016-03-15 11:42:04 CET
Added
  app_logger.warn('Could not read %s' % filename)

Writing the whole exception string can be a bit irritating. ParsingError has some special formatting.
Comment 7 Florian Best univentionstaff 2016-03-15 13:50:41 CET
(In reply to Dirk Wiesenthal from comment #6)
> Added
>   app_logger.warn('Could not read %s' % filename)
> 
> Writing the whole exception string can be a bit irritating. ParsingError has
> some special formatting.
It's not written into /var/log/univention/management-console-module-appcenter.log.
The exception details are also important.
Comment 8 Dirk Wiesenthal univentionstaff 2016-03-15 16:27:08 CET
str(exc) is now logged as debug in appcenter.log. In umc-m-appcenter.log you will find

15.03.16 14:44:18.124  MODULE      ( PROCESS ) : Could not read /var/cache/univention-appcenter/....meta

The str(exc) is not logged, because logging works via the PackageManager in UMC and it does not support debug, only PROCESS and ERROR. Should be enough in appcenter.log if we need the information.
Comment 9 Florian Best univentionstaff 2016-03-16 06:55:39 CET
(In reply to Dirk Wiesenthal from comment #8)
> str(exc) is now logged as debug in appcenter.log. In umc-m-appcenter.log you
> will find
> 
> 15.03.16 14:44:18.124  MODULE      ( PROCESS ) : Could not read
> /var/cache/univention-appcenter/....meta
> 
> The str(exc) is not logged, because logging works via the PackageManager in
> UMC and it does not support debug, only PROCESS and ERROR. Should be enough
> in appcenter.log if we need the information.
Hm. I'm still somehow unsatisfied: Now "univention-app list" tells me for 4 apps "Could not read /var/cache/univention-appcenter/$APP.meta". Are they supposed to exists or is this an error? Why don't they exists? Logging the exception as DEBUG is not really nice as it is an ERROR! If we need the debug information somewhen they are not present with the default debug level.
Comment 10 Dirk Wiesenthal univentionstaff 2016-03-16 09:13:35 CET
IOError, OSError are ignored (because the reason is probably a missing file, which may happen for .meta files (which is no real problem of the App Center, but the App Center catalogue).

ParsingError is logged as warning with complete str(exc)
Comment 11 Florian Best univentionstaff 2016-03-16 09:14:06 CET
OK: Fix
OK: YAML
Comment 12 Philipp Hahn univentionstaff 2016-03-17 14:28:55 CET
<http://errata.software-univention.de/ucs/4.1/130.html>