Bug 46821 - Opening the "App Center" is too slow
Opening the "App Center" is too slow
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: App Center
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3-1-errata
Assigned To: Dirk Wiesenthal
Felix Botner
:
Depends on: 44783
Blocks:
  Show dependency treegraph
 
Reported: 2018-04-13 13:40 CEST by Dirk Wiesenthal
Modified: 2018-06-28 10:51 CEST (History)
3 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 2: Improvement: Would be a product improvement
Who will be affected by this bug?: 4: Will affect most installed domains
How will those affected feel about the bug?: 1: Nuisance – not a big deal but noticeable
User Pain: 0.046
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional): Usability
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Wiesenthal univentionstaff 2018-04-13 13:40:02 CEST
+++ This bug was initially created as a clone of Bug #44783 +++
/var/cache/univention-appcenter/appcenter.software-univention.de/4.1/.apps.en.json
Opening the UMC module "App Center" feels very slow.
If you just have a common internet connection like 16 MBit/s it needs ages.
Comment 1 Dirk Wiesenthal univentionstaff 2018-06-20 13:56:53 CEST
Fixed in
  univention-appcenter 7.0.2-3A~4.3.0.201806201340

4393e5c1: Implemented get_all_apps() - which is key to the UMC overview of all Apps - by avoiding another round of find() for each App. find() iterates again over every single app version. get_all_apps() now has its own implementation of find() which is a bit code duplication but should not be too bad.

18ebbaac: Delay imports and initializations. For the installation of an App, this will not help in summary, but some actions do not require to go over all parts of the code. Especially this will now improve:
  ucr commit /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/default-ssl.conf /usr/share/univention-management-console/i18n/de/apps.mo /usr/share/univention-portal/apps.json

b12662e1: New initialization of App objects. This will improve loading and re-building of the cache. We keep backwards compatibility, although the old **kwargs is not used anymore in the App Center.

d95ed244: Setup the cache in the UMC module. This helps because during UMC login, one UMC request is sent to the App Center module. If the module is opened afterwards, the cache is already loaded. Otherwise we would suffer from the "delay improvements" from 18ebbaac.

We had some more tricks, but the ones used here should be clear and not harmful.

You can test the impact of the patch with:
  /usr/share/ucs-test/20_appcenter/60_check_appcenter_performance | grep "answered
in"

(ucs-test-appcenter)
Comment 2 Philipp Hahn univentionstaff 2018-06-21 09:23:36 CEST
jenkins-regression: 20_appcenter/60_check_appcenter_performance fails in all 9 scenarios:

[2018-06-21 01:13:41.624553] Appcenter update starting
(2018-06-21 01:13:41.624658) Traceback (most recent call last):
(2018-06-21 01:13:41.624689)   File "60_check_appcenter_performance", line 44, in <module>
(2018-06-21 01:13:41.624710)     main()
(2018-06-21 01:13:41.624729)   File "60_check_appcenter_performance", line 19, in main
(2018-06-21 01:13:41.624749)     appcenter_update = actions.update.Update()
(2018-06-21 01:13:41.624768) AttributeError: 'module' object has no attribute 'update'

<http://jenkins.knut.univention.de:8080/job/UCS-4.3/job/UCS-4.3-1/job/AutotestJoin/lastCompletedBuild/SambaVersion=s4,Systemrolle=slave/testReport/20_appcenter/60_check_appcenter_performance/test/>
Comment 3 Dirk Wiesenthal univentionstaff 2018-06-21 11:42:00 CEST
I fixed the import in
  ucs-test 8.0.28-140A~4.3.0.201806211112
Comment 4 Felix Botner univentionstaff 2018-06-21 16:08:46 CEST
-> python -m cProfile /usr/bin/univention-app list|grep getMachine
      225    0.006    0.000    3.623    0.016 uldap.py:160(getMachineConnection)
      225    0.009    0.000    3.617    0.016 uldap.py:91(getMachineConnection)

Patch?
diff --git a/management/univention-appcenter/python/appcenter/actions/list_apps.py b/management/univention-appcenter/python/appcenter/actions/list_apps.py
index 6ff3313..668ac81 100644
--- a/management/univention-appcenter/python/appcenter/actions/list_apps.py
+++ b/management/univention-appcenter/python/appcenter/actions/list_apps.py
@@ -37,7 +37,7 @@ from fnmatch import fnmatch
 
 from univention.appcenter.actions import UniventionAppAction
 from univention.appcenter.app_cache import Apps
-from univention.appcenter.udm import get_app_ldap_object
+from univention.appcenter.udm import get_app_ldap_object, get_machine_connection
 from univention.appcenter.utils import flatten
 from univention.appcenter.ucr import ucr_get, ucr_is_true
 
@@ -124,6 +124,7 @@ class List(UniventionAppAction):
 
        def _list(self, pattern):
                ret = []
+               lo, pos = get_machine_connection()
                for app in self.get_apps():
                        versions = []
                        installations = {}
@@ -132,7 +133,7 @@ class List(UniventionAppAction):
                                        continue
                        app = Apps().find(app.id, latest=True)
                        for _app in Apps().get_all_apps_with_id(app.id):
-                               ldap_obj = get_app_ldap_object(_app)
+                               ldap_obj = get_app_ldap_object(_app, lo=lo, pos=pos)
                                servers = ldap_obj.installed_on_servers()
                                versions.append(_app.version)
                                installations[_app.version] = servers

which reduces univention-app list from 5s to 2s
Comment 5 Felix Botner univentionstaff 2018-06-21 16:31:29 CEST
great work:

univention-app update 23s -> 13s
(empty chache)

univention-app info 2s -> 0.7s

univention-app remove wordpress 24s -> 11s

univention-app install wordpress 43s -> 27s
(docker image already pulled, database packages already installed)
Comment 6 Felix Botner univentionstaff 2018-06-21 16:32:02 CEST
(In reply to Felix Botner from comment #5)
> great work:
> 
> univention-app update 23s -> 13s
> (empty chache)
> 
> univention-app info 2s -> 0.7s
> 
> univention-app remove wordpress 24s -> 11s
> 
> univention-app install wordpress 43s -> 27s
> (docker image already pulled, database packages already installed)

waiting for tomorrows ucs-test results ...
Comment 7 Dirk Wiesenthal univentionstaff 2018-06-21 17:00:14 CEST
(In reply to Felix Botner from comment #4)
> ...
> which reduces univention-app list from 5s to 2s

Good catch!
  univention-appcenter 7.0.2-4A~4.3.0.201806211635
Comment 8 Felix Botner univentionstaff 2018-06-22 12:15:59 CEST
OK - univention-appcenter
OK - tests
OK - YAML
Comment 9 Erik Damrose univentionstaff 2018-06-28 10:51:11 CEST
<http://errata.software-univention.de/ucs/4.3/126.html>