Bug 34834 - Installation got stuck in 35univention-management-console-module-appcenter.inst
Installation got stuck in 35univention-management-console-module-appcenter.inst
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - App-Center
UCS 3.2
Other Linux
: P5 normal (vote)
: UCS 3.2-2
Assigned To: Stefan Gohmann
Philipp Hahn
http://tools.ietf.org/html/rfc2616#se...
:
Depends on: 32935
Blocks: 34804
  Show dependency treegraph
 
Reported: 2014-05-14 15:42 CEST by Stefan Gohmann
Modified: 2014-09-01 11:35 CEST (History)
6 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2014-05-14 15:42:52 CEST
> During the Release-tests for UCS-3.2-2 several (2+) VMs got stuck during the
> /usr/sbin/univention-register-apps; one has 120 open TCP+SSL connections to
> <https://download2.software-univention.de/>. The VM is stuck since >20h. 


+++ This bug was initially created as a clone of Bug #32935 +++

The app_center.py-module currently starts one thread for each file to be downloaded.
This leads to (currently) 209 threads with 209 TCP-connections (trying to) starting 209 Apache processes on the server.
(Each with a TCP and HTTPS handshake)

Every added App increases this problem.

As these files are static a single connection with properly pipelined requests is the fastest and most efficient (for both client and server).

Emulating this process with wget allows fetching all 209 files in just 4.9 seconds using a single connection, whereas using 209 connections requires 6.5 seconds (on a quad-core machine) but using about 100 times more CPU time.

(I think wget does not do pipelining so the runtime gains should be even greater)


Not using threads would also allow the code to do error handling (currently missing files are only logged, not handled)


This may impact the release negatively because of the release many new installations at the same time.
Comment 1 Stefan Gohmann univentionstaff 2014-05-14 15:50:02 CEST
A timeout has been added to univention-register-apps: r50256

Revert of Bug #32935: r50255

Changelog: r50257
Comment 2 Philipp Hahn univentionstaff 2014-05-14 21:20:19 CEST
OK: revert r50255
OK: alarm r50256
OK: changelog r50257

OK: tc qdisc ... tc class ... tc filter ... univention-register-apps

FYI: In once test I received the very long error output like:
...
14.05.14 15:14:18.307 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/univention-demoapp.png: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.307 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/samba-memberserver.ini: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.308 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/samba-memberserver.png: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.308 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/zarafa_20140206.ini: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.308 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/zarafa_20140206.png: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.308 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/univention-repository/3.2/maintained/component/zarafa_20140206/README_UPDATE: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.308 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/univention-repository/3.2/maintained/component/uccbeta_20140501/README_EN: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.308 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/univention-repository/3.2/maintained/component/uccbeta_20140501/README_DE: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.309 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/uccbeta_20140501.png: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.309 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/uccbeta_20140501.ini: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.309 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/oxseforucs_20131122.ini: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.309 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/oxseforucs_20131122.png: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.309 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/univention-repository/3.2/maintained/component/sugarcrm_20140310/LICENSE_AGREEMENT: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.309 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/sugarcrm_20140310.ini: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.310 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/meta-inf/3.2/sugarcrm_20140310.png: <urlopen error [Errno 101] Network is unreachable>
14.05.14 15:14:18.310 MODULE (ERROR): Error downloading https://appcenter.software-univention.de/univention-repository/3.2/maintained/component/sugarcrm_20140310/README_UPDATE: <urlopen error [Errno 101] Network is unreachable>
...
After that the process could not even be terminated by SIGINT. SIGTERM did it.
Comment 3 Stefan Gohmann univentionstaff 2014-05-20 07:53:29 CEST
UCS 3.2-2 has been released:
 http://docs.univention.de/release-notes-3.2-2-en.html
 http://docs.univention.de/release-notes-3.2-2-de.html

If this error occurs again, please use "Clone This Bug".