Bug 43847 - App Center update performance
App Center update performance
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: App Center
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.2-1-errata
Assigned To: Dirk Wiesenthal
Felix Botner
:
Depends on: 43662 45085
Blocks: 45080
  Show dependency treegraph
 
Reported: 2017-03-14 16:38 CET by Stefan Gohmann
Modified: 2017-08-26 20:36 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 2: Improvement: Would be a product improvement
Who will be affected by this bug?: 5: Will affect all installed domains
How will those affected feel about the bug?: 1: Nuisance – not a big deal but noticeable
User Pain: 0.057
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2017-03-14 16:38:57 CET
I don't think it makes sense that we check every single file in the archive. I think we should add some files with a signature and extract them without re-checking every single file.


+++ This bug was initially created as a clone of Bug #43662 +++

I've switched to our test app center. It took more than 2 minutes to update the internal cache:

 12627 actions.update.progress          17-02-27 20:58:47 [   DEBUG]: 0
[...]
 12627 actions.update                   17-02-27 21:01:20 [   DEBUG]: Extracting wildfly_20170222160423.meta
 12627 actions.update                   17-02-27 21:01:20 [   DEBUG]: Extracting wildfly_20170222160423.inst
 12627 actions.update                   17-02-27 21:01:20 [   DEBUG]: Extracting wildfly_20170222160423.ini
 12627 actions.update                   17-02-27 21:01:20 [   DEBUG]: Extracting wildfly_20170222160423.README_EN
 12627 actions.update                   17-02-27 21:01:20 [   DEBUG]: Extracting wildfly_20170222160423.logo
 12627 cache                            17-02-27 21:01:20 [   DEBUG]: Cache outdated. Need to rebuild
 12627 cache                            17-02-27 21:01:21 [   DEBUG]: Saved 21 apps into cache
 12627 cache                            17-02-27 21:01:21 [   DEBUG]: Cache outdated. Need to rebuild
 12627 cache                            17-02-27 21:01:23 [   DEBUG]: Saved 335 apps into cache
 12627 actions.update                   17-02-27 21:01:23 [   DEBUG]: Updating app files...
 12627 actions.update                   17-02-27 21:01:23 [   DEBUG]: Updating app icon files in UMC directory...
 12627 actions.update                   17-02-27 21:01:23 [    INFO]: File: /usr/share/univention-management-console/modules/apps.xml
 12627 actions.update                   17-02-27 21:01:23 [    INFO]: 
 12627 actions.update                   17-02-27 21:01:24 [    INFO]: File: /usr/share/univention-management-console/i18n/de/apps.mo
 12627 actions.update                   17-02-27 21:01:24 [    INFO]: 
 12627 actions.update                   17-02-27 21:01:25 [   DEBUG]: copying /var/cache/univention-appcenter/appcenter-test.software-univention.de/4.2/adconnector.logo -> /usr/share/univention-management-console-frontend/js/dijit/themes/umc/icons/scalable/apps-adconnector.svg
Comment 1 Dirk Wiesenthal univentionstaff 2017-07-27 11:44:35 CEST
We decided to do it with zsync.

 * all.tar.gz will be downloaded via zsync
   => This should vastly speed up download time as only the changed portion of the archive needs to be downloaded (+ minimal overhead)
 * all.tar.gz.gpg will be used
   => This allows us to: ↴
 * Archive is extracted as is and without any further checks
   => This will greatly increase unpack speed; we will use the tar binary instead of Python tarfile
Comment 2 Dirk Wiesenthal univentionstaff 2017-07-27 13:04:41 CEST
I have changed ucs-test in 4.1-4 and 4.2-1 referring to this bug.
Comment 3 Dirk Wiesenthal univentionstaff 2017-07-27 13:25:47 CEST
Fixed in
  univention-appcenter 6.0.8-17A~4.2.0.201707271323
Comment 4 Felix Botner univentionstaff 2017-08-07 10:50:33 CEST
looks good
OK - jenkins tests (i updated the update threshold in 
     20_appcenter/60_check_appcenter_performance to 15s)
OK - test
OK - appcenter/test appcenter
OK - YAML
Comment 5 Arvid Requate univentionstaff 2017-08-09 16:57:19 CEST
<http://errata.software-univention.de/ucs/4.2/127.html>
Comment 6 Michael Voigt 2017-08-26 20:36:37 CEST
I believe that this problem is not solved.
Current UCS version is 4.2-1 errata144
When I install an APP, it takes several minutes.
One interesting thing is that if repository / online / component / cool-solutions: yes an "Internal Server Error"
If repository / online / component / cool-solutions: no the process runs well, but takes several minutes.

The message that the cache needs to be rebuild does not disappear.

2017-08-26 15: 08: 15,091 DEBUG: univention.appcenter.actions.update.progress: 100
Cache outdated. Need to rebuild
2017-08-26 15: 08: 15,092 DEBUG: univention.appcenter.cache: Cache outdated. Need to rebuild
Loaded 74 apps from cache
2017-08-26 15: 08: 15,140 DEBUG: univention.appcenter.cache: Loaded 74 apps from cache
Cache outdated. Need to rebuild
2017-08-26 15: 08: 15,140 DEBUG: univention.appcenter.cache: Cache outdated. Need to rebuild
Loaded 325 apps from cache
2017-08-26 15: 08: 15,326 DEBUG: univention.appcenter.cache: Loaded 325 apps from cache