Bug 34989 - App Center repository creation is too slow
App Center repository creation is too slow
Status: RESOLVED DUPLICATE of bug 33035
Product: UCS
Classification: Unclassified
Component: UMC - App-Center
UCS 3.2
Other Linux
: P5 normal (vote)
: ---
Assigned To: UMC maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-05-27 12:17 CEST by Dirk Wiesenthal
Modified: 2014-07-21 08:29 CEST (History)
3 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Error handling
Max CVSS v3 score:


Attachments
Using 25_ucs-appcenter.list instead (13.80 KB, patch)
2014-06-10 14:30 CEST, Dirk Wiesenthal
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Wiesenthal univentionstaff 2014-05-27 12:17:53 CEST
Checking whether the app can be installed requires two ucr commits on /etc/apt/sources.list. This requires so much time that slow internet connection cause errors - leaving the app uninstallable/unupgradable.

Reported by user feedback.

 Traceback:
Unrecoverable error in the server.
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/cherrypy/_cpwsgi.py", line 79, in setapp
    s, h, b = self.get_response()
  File "/usr/lib/pymodules/python2.6/cherrypy/_cpwsgi.py", line 219, in get_response
    response = self.request.run(meth, path, qs, rproto, headers, rfile)
  File "/usr/lib/pymodules/python2.6/cherrypy/_cprequest.py", line 567, in run
    raise cherrypy.TimeoutError()
TimeoutError
Comment 1 Florian Best univentionstaff 2014-05-27 12:21:39 CEST
Reported 3 times via UMC-feedback.
Comment 2 Dirk Wiesenthal univentionstaff 2014-05-27 12:23:38 CEST
One idea might be to add
  /etc/apt/sources.list.d/25_ucs_appcenter.list

where no checks are done, it just dumps the UCR variables.

could be filtered by
repository/online/component/ucsschool_20140131/appcenter: true

Needs to make sure that major upgrades are forbidden as long as the component is not available for the new version.

It may be necessary to change the UCR variables to

repository/online/appcenter/ucsschool_20140131

instead to not trigger the normal ucr commits.



Another solution may be speeding up the overall process in /etc/apt/sources.list.d/*. But I guess the expensive checks are there for a reason.
Comment 3 Alexander Kläser univentionstaff 2014-05-27 16:59:10 CEST
(In reply to Dirk Wiesenthal from comment #0)
> Checking whether the app can be installed requires two ucr commits on
> /etc/apt/sources.list. This requires so much time that slow internet
> connection cause errors - leaving the app uninstallable/unupgradable.

Does apt-get respect the HTTP caching capabilities?
Comment 4 Dirk Wiesenthal univentionstaff 2014-06-10 14:30:35 CEST
Created attachment 5945 [details]
Using 25_ucs-appcenter.list instead

Super fast version:

Using repository/app_center/apps/.* to populate /etc/apt/sources.list.d/25_ucs-appcenter.list

No checks for the repository are done. Just writes it down if the app is found.

Pro:
Blazingly fast
Possible to switch between the old and new format

Con:
No checks for existence. Gives 404 error if repository does not exist!
  => /all, /i386 and /amd64 need to exist (at least empty)!
Not listed in the Repository Settings tab of the App Center module (may be a Pro, though...)
No possibility to adjust it except something like 'version=current' and 'version=""'
  => All apps now have the same server. No possibility to have some from main and some from test App Center.
  => May be a problem if we ever allow "local app center" with proxy, username, password and stuff.

Caveats:
Currently not possible to check whether an app is available in UCS 3.2+ and stop upgrade if not.
Seems there is no real "hook" in the updater code that could be called from the App Center. Needs adjustment there. univention-updater/modules/univention/updater/tools.py "def release_update_available" might be the right place.

Todo:
After applying the patch and installing the new package, run univention-migrate-apps.
  => Needs to go to postinst eventually
Some apps do not have /all /i386 and /amd64. They need to be added, until then those cannot be tested:
  agorumcore-pro
  owncloud
  owncloud5
  owncloud6
  sugarcrm
  univention-demoapp
If this patch is ever released, make sure that each future app repository has these directories.
You may go back with:
  ucr set repository/app_center/new_format=false
  univention-migrate-apps --to-old
  -> This has not really been tested, though...
Comment 5 Stefan Gohmann univentionstaff 2014-06-17 08:06:20 CEST
I think we should fix this via Bug #33035. If it is not possible we should re-check this issue.

*** This bug has been marked as a duplicate of bug 33035 ***