Bug 38345

Summary: App Center startup performance
Product: UCS Reporter: Stefan Gohmann <gohmann>
Component: UMC - App-CenterAssignee: Alexander Kramer <kramer>
Status: CLOSED FIXED QA Contact: Florian Best <best>
Severity: normal    
Priority: P5 CC: best, gulden, klaeser, kramer, walkenhorst
Version: UCS 4.0   
Target Milestone: UCS 4.0-2-errata   
Hardware: Other   
OS: Linux   
What kind of report is it?: --- What type of bug is this?: ---
Who will be affected by this bug?: --- How will those affected feel about the bug?: ---
User Pain: Enterprise Customer affected?:
School Customer affected?: ISV affected?:
Waiting Support: Flags outvoted (downgraded) after PO Review:
Ticket number: Bug group (optional):
Max CVSS v3 score:
Attachments: management-console-module-appcenter.log
WIP patch: ucr.load and getMachineConnection

Description Stefan Gohmann univentionstaff 2015-04-24 10:45:54 CEST
Currently, it took between 5 and 9 seconds to open the App Center. I think that is too long.
Comment 1 Nico Gulden univentionstaff 2015-05-18 10:53:15 CEST
I also experienced that long loading time during demonstration. I used demo.univention.com.
Comment 2 Alexander Kramer univentionstaff 2015-06-05 14:43:46 CEST
Created attachment 6943 [details]
management-console-module-appcenter.log

Maybe we can also adapt the PackageManager __init__ to get a better performance
Comment 3 Alexander Kramer univentionstaff 2015-06-05 14:49:00 CEST
Created attachment 6944 [details]
WIP patch: ucr.load and getMachineConnection

in app_center.py to_dict():

ucr.load() and
ldap_object = self.get_ldap_object() --> admin_uldap.getMachineConnection()

Both methods are slowing down the performance, so we added two helper functions and global variables to check if we need to
a) reload ucr
b) establish a new ldap connection

pls note wip for this patch
Comment 4 Alexander Kläser univentionstaff 2015-06-05 21:47:34 CEST
@Alex: Cool :) ! (BTW, diff -Nu yields a more common diff format.)

@Florian: The LDAP timeout should be caught in uldap directly, is that correct?
Comment 5 Florian Best univentionstaff 2015-06-08 10:19:45 CEST
(In reply to Alexander Kläser from comment #4)
> @Alex: Cool :) ! (BTW, diff -Nu yields a more common diff format.)
> 
> @Florian: The LDAP timeout should be caught in uldap directly, is that
> correct?
No, the methods will raise univention.admin.uexceptions.{ldapTimeout,ldapError} when a timeout occurs.
I think we have to implement the same workarounds as in UDM, too.
Comment 6 Alexander Kramer univentionstaff 2015-07-07 14:54:32 CEST
r61838
r61840

We improved the performance by using decorators for loading ucr (inside to_dict) and requesting a ldap connection/position.

As discussed with Florian later on we can remove the decorators.py file and use a more generic solution.

We also register a umcCommand (appcenter/ping) on the umc startup for another 700milliseconds.

Package: univention-management-console-module-appcenter
Version: 4.1.20-33.373.201507071447
Branch: ucs_4.0-0
Scope: errata4.0-2

r61843
yaml file
Comment 7 Alexander Kramer univentionstaff 2015-07-07 15:46:29 CEST
r61847
univention-management-console-module-appcenter (4.1.20-34) 
* Bug #38345: Adapted decorators


Package: univention-management-console-module-appcenter
Version: 4.1.20-34.374.201507071530
Branch: ucs_4.0-0
Scope: errata4.0-2

r61849
yaml file
Comment 8 Florian Best univentionstaff 2015-07-08 14:40:41 CEST
*** Bug 38860 has been marked as a duplicate of this bug. ***
Comment 9 Florian Best univentionstaff 2015-07-09 11:13:48 CEST
Please add a preinst like management/univention-directory-manager-modules/debian/python-univention-directory-manager.preinst to prevent problems during upgrade.

Please fix Bug #38860.
Comment 10 Florian Best univentionstaff 2015-07-10 10:54:11 CEST
Please also alter util.py.

I had my fingers in the code, so I better don't to the QA?!
Comment 11 Alexander Kramer univentionstaff 2015-07-10 13:02:57 CEST
r62036

Added error handling for 38860
Added a preinst
Adapted util.py

Successful build
Package: univention-management-console-module-appcenter
Version: 4.1.20-35.376.201507101257
Branch: ucs_4.0-0
Scope: errata4.0-2

r62037
yaml file
Comment 12 Florian Best univentionstaff 2015-07-14 09:48:54 CEST
Please adjust the following thing in util.py:
IOError is not raised by the decorators/functions instead lo is None in that case.
Comment 13 Florian Best univentionstaff 2015-07-14 09:55:55 CEST
(In reply to Florian Best from comment #12)
> Please adjust the following thing in util.py:
> IOError is not raised by the decorators/functions instead lo is None in that
> case.
Also there is univention.uldap not univention.admin.uldap used. You have to add 'lo = lo.lo'.
Comment 14 Florian Best univentionstaff 2015-07-14 12:54:03 CEST
*** Bug 38914 has been marked as a duplicate of this bug. ***
Comment 15 Alexander Kramer univentionstaff 2015-07-15 08:45:36 CEST
(In reply to Florian Best from comment #12)
> Please adjust the following thing in util.py:
> IOError is not raised by the decorators/functions instead lo is None in that
> case.

r62112
Adjusted the requested parts in util.py
Also removed needless import from appcenter, decorator and util.py
Tested it localy on my vm - should work fine.

r62113
Adapted changelog


Successful build
Package: univention-management-console-module-appcenter
Version: 4.1.20-37.381.201507150840
Branch: ucs_4.0-0
Scope: errata4.0-2

r62114
yaml file
Comment 16 Alexander Kramer univentionstaff 2015-07-15 09:36:10 CEST
r62115
Merged the changes into 4.1
Comment 17 Alexander Kramer univentionstaff 2015-07-15 12:18:03 CEST
r62119
As discussed a added error handling if lo is None

Package: univention-management-console-module-appcenter
Version: 4.1.20-38.382.201507151214
Branch: ucs_4.0-0
Scope: errata4.0-2

r62121
yaml file

r62120
merged changes to 4.1-0
Comment 18 Florian Best univentionstaff 2015-07-30 13:14:33 CEST
(In reply to Florian Best from comment #9)
> Please add a preinst like
> management/univention-directory-manager-modules/debian/python-univention-
> directory-manager.preinst to prevent problems during upgrade.
> 
> Please fix Bug #38860.

The preinst is for the wrong package. It has to be for univention-management-console-module-appcenter not univention-management-console-module-apps.
Comment 19 Alexander Kramer univentionstaff 2015-07-30 14:30:44 CEST
(In reply to Florian Best from comment #18)
> (In reply to Florian Best from comment #9)
> > Please add a preinst like
> > management/univention-directory-manager-modules/debian/python-univention-
> > directory-manager.preinst to prevent problems during upgrade.
> > 
> > Please fix Bug #38860.
> 
> The preinst is for the wrong package. It has to be for
> univention-management-console-module-appcenter not
> univention-management-console-module-apps.

As requested I added a preinst file for module-appcenter and moved the code from 
r62036

Also added 
dpkg --compare-versions "$2" 4.1.20-41
r62533

And removed two typos
r62534

Successful build
Package: univention-management-console-module-appcenter
Version: 4.1.20-43.388.201507301424
Branch: ucs_4.0-0
Scope: errata4.0-2

yaml file r62535
Comment 20 Florian Best univentionstaff 2015-07-30 15:24:03 CEST
OK: 3.8 seconds to open the appcenter now. It could be further improved (to 2 seconds) if we make a appcenter/query after UMC login.
YAML: OK
OK: Merge to UCS 4.1 (the preinst is not required in 4.1)
Comment 21 Janek Walkenhorst univentionstaff 2015-08-06 17:54:05 CEST
<http://errata.univention.de/ucs/4.0/267.html>