Univention Bugzilla – Bug 43847
App Center update performance
Last modified: 2017-08-26 20:36:37 CEST
I don't think it makes sense that we check every single file in the archive. I think we should add some files with a signature and extract them without re-checking every single file. +++ This bug was initially created as a clone of Bug #43662 +++ I've switched to our test app center. It took more than 2 minutes to update the internal cache: 12627 actions.update.progress 17-02-27 20:58:47 [ DEBUG]: 0 [...] 12627 actions.update 17-02-27 21:01:20 [ DEBUG]: Extracting wildfly_20170222160423.meta 12627 actions.update 17-02-27 21:01:20 [ DEBUG]: Extracting wildfly_20170222160423.inst 12627 actions.update 17-02-27 21:01:20 [ DEBUG]: Extracting wildfly_20170222160423.ini 12627 actions.update 17-02-27 21:01:20 [ DEBUG]: Extracting wildfly_20170222160423.README_EN 12627 actions.update 17-02-27 21:01:20 [ DEBUG]: Extracting wildfly_20170222160423.logo 12627 cache 17-02-27 21:01:20 [ DEBUG]: Cache outdated. Need to rebuild 12627 cache 17-02-27 21:01:21 [ DEBUG]: Saved 21 apps into cache 12627 cache 17-02-27 21:01:21 [ DEBUG]: Cache outdated. Need to rebuild 12627 cache 17-02-27 21:01:23 [ DEBUG]: Saved 335 apps into cache 12627 actions.update 17-02-27 21:01:23 [ DEBUG]: Updating app files... 12627 actions.update 17-02-27 21:01:23 [ DEBUG]: Updating app icon files in UMC directory... 12627 actions.update 17-02-27 21:01:23 [ INFO]: File: /usr/share/univention-management-console/modules/apps.xml 12627 actions.update 17-02-27 21:01:23 [ INFO]: 12627 actions.update 17-02-27 21:01:24 [ INFO]: File: /usr/share/univention-management-console/i18n/de/apps.mo 12627 actions.update 17-02-27 21:01:24 [ INFO]: 12627 actions.update 17-02-27 21:01:25 [ DEBUG]: copying /var/cache/univention-appcenter/appcenter-test.software-univention.de/4.2/adconnector.logo -> /usr/share/univention-management-console-frontend/js/dijit/themes/umc/icons/scalable/apps-adconnector.svg
We decided to do it with zsync. * all.tar.gz will be downloaded via zsync => This should vastly speed up download time as only the changed portion of the archive needs to be downloaded (+ minimal overhead) * all.tar.gz.gpg will be used => This allows us to: ↴ * Archive is extracted as is and without any further checks => This will greatly increase unpack speed; we will use the tar binary instead of Python tarfile
I have changed ucs-test in 4.1-4 and 4.2-1 referring to this bug.
Fixed in univention-appcenter 6.0.8-17A~4.2.0.201707271323
looks good OK - jenkins tests (i updated the update threshold in 20_appcenter/60_check_appcenter_performance to 15s) OK - test OK - appcenter/test appcenter OK - YAML
<http://errata.software-univention.de/ucs/4.2/127.html>
I believe that this problem is not solved. Current UCS version is 4.2-1 errata144 When I install an APP, it takes several minutes. One interesting thing is that if repository / online / component / cool-solutions: yes an "Internal Server Error" If repository / online / component / cool-solutions: no the process runs well, but takes several minutes. The message that the cache needs to be rebuild does not disappear. 2017-08-26 15: 08: 15,091 DEBUG: univention.appcenter.actions.update.progress: 100 Cache outdated. Need to rebuild 2017-08-26 15: 08: 15,092 DEBUG: univention.appcenter.cache: Cache outdated. Need to rebuild Loaded 74 apps from cache 2017-08-26 15: 08: 15,140 DEBUG: univention.appcenter.cache: Loaded 74 apps from cache Cache outdated. Need to rebuild 2017-08-26 15: 08: 15,140 DEBUG: univention.appcenter.cache: Cache outdated. Need to rebuild Loaded 325 apps from cache 2017-08-26 15: 08: 15,326 DEBUG: univention.appcenter.cache: Loaded 325 apps from cache