Bug 34543 - BadStatusLine exception in appcenter not handled
Summary: BadStatusLine exception in appcenter not handled
Status: CLOSED FIXED
Alias: None
Product: UCS
Classification: Unclassified
Component: UMC - App-Center
Version: UCS 4.1
Hardware: Other Linux
: P5 normal
Target Milestone: UCS 4.2-0-errata
Assignee: Felix Botner
QA Contact: Dirk Wiesenthal
URL:
Keywords:
Depends on:
Blocks: 45444
  Show dependency tree / graph
 
Reported: 2014-04-14 14:24 CEST by Dirk Wiesenthal
Modified: 2017-10-12 20:36 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.171
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2017093021000259, 2015031621000237, 2015090221000341, 2015102821000013, 2016052321000094, 2016052721000274, 2016090721000411, 2016092021000387, 2016092921000683, 2016120721000664, 2017030321000454, 2017033021000528, 2017050421000332, 2017061421000544
Bug group (optional): Error handling, External feedback
Customer ID:
Max CVSS v3 score:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Wiesenthal univentionstaff 2014-04-14 14:24:58 CEST 
Reported by traceback feedback:

 Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/__init__.py",
line 204, in execute
    func( request )
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py",
line 305, in _response
    result = _multi_response(self, request)
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py",
line 432, in _response
    for res in function(self, iterator, *nones):
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py",
line 271, in _fake_func
    yield function(self, *args)
  File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/__init__.py",
line 101, in query
    applications = Application.all(force_reread=True)
  File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/app_center.py",
line 606, in all
    cls._get_category_translations(fake=not localize)
  File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/app_center.py",
line 484, in _get_category_translations
    fp = urlopen(url)
  File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/util.py",
line 96, in urlopen
    return urllib2.urlopen(request)
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1192, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1156, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
  File "/usr/lib/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
  File "/usr/lib/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
  File "/usr/lib/python2.6/httplib.py", line 739, in send
    self.connect()
  File "/usr/lib/python2.6/httplib.py", line 1115, in connect
    self._tunnel()
  File "/usr/lib/python2.6/httplib.py", line 706, in _tunnel
    (version, code, message) = response._read_status()
  File "/usr/lib/python2.6/httplib.py", line 355, in _read_status
    raise BadStatusLine(line)
BadStatusLine


Code from /usr/lib/python2.6/httplib.py

 352  if not line:
 353    # Presumably, the server closed the connection before
 354    # sending a valid response.
 355    raise BadStatusLine(line)
Comment 1 Dirk Wiesenthal univentionstaff 2014-04-29 10:28:19 CEST 
Also traceback feedback

May also happen in other situations (u = UniventionUpdater()):

 Traceback:
The init function of the module has failed: BadStatusLine:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py",
line 228, in handle
    self.__handler.init()
  File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/__init__.py",
line 88, in init
    self.uu = UniventionUpdater(False)
  File "/usr/lib/pymodules/python2.6/univention/updater/tools.py", line 556, in __init__
    self.ucr_reinit()
  File "/usr/lib/pymodules/python2.6/univention/updater/tools.py", line 611, in ucr_reinit
    assert self.server.access('/univention-repository/')
  File "/usr/lib/pymodules/python2.6/univention/updater/tools.py", line 437, in access
    res = UCSHttpServer.opener.open(req)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1184, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1157, in do_open
    r = h.getresponse()
  File "/usr/lib/python2.6/httplib.py", line 990, in getresponse
    response.begin()
  File "/usr/lib/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.6/httplib.py", line 355, in _read_status
    raise BadStatusLine(line)
BadStatusLine
Comment 2 Florian Best univentionstaff 2015-03-17 10:05:13 CET 
Traceback from comment #1 was reported again, 4.0-1 errata113 (Walle)
Comment 3 Florian Best univentionstaff 2015-09-02 14:59:35 CEST 
Execution of command 'appcenter/invoke_dry_run' has failed:

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 282, in execute
    function(self, request)
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 81, in _decorated
    return func(self, request, *a, **kwargs)
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 202, in invoke_dry_run
    self.invoke(request)
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 81, in _decorated
    return func(self, request, *a, **kwargs)
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 188, in _response
    return function(self, request)
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 229, in invoke
    application = Application.find(application_id)
  File "%PY2.7%/univention/management/console/modules/appcenter/app_center.py", line 537, in find
    for application in cls.all():
  File "%PY2.7%/univention/management/console/modules/appcenter/app_center.py", line 763, in all
    cls._get_category_translations(fake=not localize)
  File "%PY2.7%/univention/management/console/modules/appcenter/app_center.py", line 551, in _get_category_translations
    fp = urlopen(url)
  File "%PY2.7%/univention/management/console/modules/appcenter/util.py", line 166, in urlopen
    return urllib2.urlopen(request, timeout=60)
  File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 401, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 419, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1219, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1034, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 371, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''


Remark:
errors

Version:
4.0-3 errata285 (Walle)
Comment 4 Florian Best univentionstaff 2015-11-16 08:46:23 CET 
Reported again, 4.0-3 errata313 (Walle)
Comment 5 Alexander Kläser univentionstaff 2016-05-23 10:51:39 CEST 
Reported again when opening the repository settings module.
Ticket#2016052321000094

UCS Version: 4.1-1 errata176 (Vahr)
 
Traceback (most recent call last):
 
BadStatusLine: ''
    raise BadStatusLine(line)
  File "/usr/lib/python2.7/httplib.py", line 371, in _read_status
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 1034, in getresponse
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1211, in http_open
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 419, in _open
    response = self._open(req, data)
 File "/usr/lib/python2.7/urllib2.py", line 401, in open
    res = UCSHttpServer.opener.open(req, timeout=self.timeout)
  File "/usr/lib/pymodules/python2.7/univention/updater/tools.py", line 424, in access
    assert self.server.access(None, '/univention-repository/')
  File "/usr/lib/pymodules/python2.7/univention/updater/tools.py", line 651, in ucr_reinit
    self.ucr_reinit()
  File "/usr/lib/pymodules/python2
Comment 6 Florian Best univentionstaff 2016-06-13 07:53:18 CEST 
Reported again, 4.1-2 errata183 (Vahr)
Comment 7 Florian Best univentionstaff 2016-09-13 11:56:18 CEST 
Reported again, 4.1-3 errata239 (Vahr)

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/protocol/modserver.py", line 271, in handle
    self.__handler.init()
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 148, in init
    self.uu = UniventionUpdater(False)
  File "%PY2.7%/univention/updater/tools.py", line 579, in __init__
    self.ucr_reinit()
  File "%PY2.7%/univention/updater/tools.py", line 660, in ucr_reinit
    assert self.server.access(None, '')
  File "%PY2.7%/univention/updater/tools.py", line 424, in access
    res = UCSHttpServer.opener.open(req, timeout=self.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 401, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 419, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1219, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1034, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 371, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''
Comment 8 Florian Best univentionstaff 2016-09-20 12:30:20 CEST 
Version: 3.2-8 errata445 (Borgfeld)

Remark: Fehler unbekannt. Abrufen von apt-get Befehlen schlägt ebenfalls größtenteils fehl

Traceback(cb8ca57701c57e246068b2ff811716c6):
Die Ausführung des Kommandos appcenter/query ist fehlgeschlagen:

Traceback (most recent call last):
  File "%PY2.6%/univention/management/console/modules/__init__.py", line 218, in execute
    func( request )
  File "%PY2.6%/univention/management/console/modules/decorators.py", line 305, in _response
    result = _multi_response(self, request)
  File "%PY2.6%/univention/management/console/modules/decorators.py", line 432, in _response
    for res in function(self, iterator, *nones):
  File "%PY2.6%/univention/management/console/modules/decorators.py", line 271, in _fake_func
    yield function(self, *args)
  File "%PY2.6%/univention/management/console/modules/appcenter/__init__.py", line 101, in query
    applications = Application.all(force_reread=True)
  File "%PY2.6%/univention/management/console/modules/appcenter/app_center.py", line 726, in all
    cls._get_category_translations(fake=not localize)
  File "%PY2.6%/univention/management/console/modules/appcenter/app_center.py", line 536, in _get_category_translations
    fp = urlopen(url)
  File "%PY2.6%/univention/management/console/modules/appcenter/util.py", line 126, in urlopen
    return urllib2.urlopen(request, timeout=60)
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1192, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1156, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
  File "/usr/lib/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
  File "/usr/lib/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
  File "/usr/lib/python2.6/httplib.py", line 739, in send
    self.connect()
  File "/usr/lib/python2.6/httplib.py", line 1115, in connect
    self._tunnel()
  File "/usr/lib/python2.6/httplib.py", line 706, in _tunnel
    (version, code, message) = response._read_status()
  File "/usr/lib/python2.6/httplib.py", line 355, in _read_status
    raise BadStatusLine(line)
BadStatusLine
Comment 9 Florian Best univentionstaff 2016-10-04 12:32:04 CEST 
Version: 4.1-3 errata282 (Vahr)

Execution of command 'appcenter/query' has failed:

Traceback (most recent call last):
  File "%PY2.7%/univention/management/console/base.py", line 283, in execute
    function(self, request)
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 318, in _response
    result = _multi_response(self, request)
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 462, in _response
    return list(function(self, iterator, *nones))
  File "%PY2.7%/univention/management/console/modules/decorators.py", line 284, in _fake_func
    yield function(self, *args)
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 182, in query
    self.update_applications()
  File "%PY2.7%/univention/management/console/modules/appcenter/__init__.py", line 196, in update_applications
    update.call_safe()
  File "%PY2.7%/univention/appcenter/actions/__init__.py", line 175, in call_safe
    return cls.call(**kwargs)
  File "%PY2.7%/univention/appcenter/actions/__init__.py", line 183, in call
    return obj.call_with_namespace(namespace)
  File "%PY2.7%/univention/appcenter/actions/__init__.py", line 189, in call_with_namespace
    result = self.main(namespace)
  File "%PY2.7%/univention/appcenter/actions/update.py", line 77, in main
    self._download_supra_files()
  File "%PY2.7%/univention/appcenter/actions/__init__.py", line 65, in _func
    return func(*args, **kwargs)
  File "%PY2.7%/univention/appcenter/actions/update.py", line 146, in _download_supra_files
    _download_supra_file('index.json.gz', version_specific=True)
  File "%PY2.7%/univention/appcenter/actions/update.py", line 133, in _download_supra_file
    response = urlopen(request)
  File "%PY2.7%/univention/appcenter/utils.py", line 250, in urlopen
    return urllib2.urlopen(request, timeout=60)
  File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 401, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 419, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "%PY2.7%/univention/appcenter/utils.py", line 237, in https_open
    return self.do_open(HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1072, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 408, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 372, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''
Comment 10 Stefan Gohmann univentionstaff 2016-10-05 10:44:37 CEST 
Do we know in which situation this happens?
Comment 11 Florian Best univentionstaff 2016-10-20 17:31:27 CEST 
(In reply to Stefan Gohmann from comment #10)
> Do we know in which situation this happens?
When the appcenter server closes the connection immediately (socket.read(x) == EOF == ''). E.g. due to high load.

This was at least 10 times reported and is therefore one of the most reported tracebacks we have.
Comment 12 Florian Best univentionstaff 2016-12-09 17:29:05 CET 
Reported again, 4.1-4 errata353 (Vahr)

Remark: Neuen Member Server angelegt. 2 Netzwerkanschlüsse mittles Bonding verbunden über UCM Netzwerk Modul
Comment 13 Florian Best univentionstaff 2017-04-21 11:45:01 CEST 
Reported again, 4.1-4 errata404 (Vahr)
Comment 14 Florian Best univentionstaff 2017-04-21 11:50:48 CEST 
Remark: Das passiert bei einem schlecht konfigurierten Proxy... also einfach nur Squid auf einem Member installieren und auf dem Master ucr set proxy/http auf den Member setzen.
Comment 15 Florian Best univentionstaff 2017-05-05 15:34:57 CEST 
Version: 4.1-4 errata408 (Vahr)
Comment 16 Felix Botner univentionstaff 2017-05-17 14:35:47 CEST 
catch httplib.BadStatusLine in possible_network_error and give a rather neutral error message

replaced all UMC_CommandError with UMC_Error in umc/python/appcenter/__init__.py

univention-appcenter: r79351
Comment 17 Dirk Wiesenthal univentionstaff 2017-05-22 12:01:28 CEST 
Works great
Comment 18 Janek Walkenhorst univentionstaff 2017-05-24 10:47:57 CEST 
<http://errata.software-univention.de/ucs/4.2/22.html>
Comment 19 Florian Best univentionstaff 2017-06-16 18:43:05 CEST 
Reported again, 4.1-4 errata420 (Vahr)
Comment 20 Florian Best univentionstaff 2017-10-12 20:36:46 CEST 
Version: 4.1-4 errata420 (Vahr)