Univention Bugzilla – Bug 42265
Replace umc_frontend_new_hash logic of time stamped symlinks with improved HTTP caching configuration
Last modified: 2017-04-04 18:28:15 CEST
Up to now, we were using symlinks with timestamps to the JavaScript in order to avoid caching problems with browsers. For UCS 4.2 we will remove the symlinks and implement proper HTTP caching configuration , instead. The motivation for this is: * The current mechanism caused some troubles in usage of UMC (e.g., Bug 42095, Bug 37565, Bug 39578). * For UCS 4.2, we would need to change the timestamp in multiple HTML sites of different packages. * The current mechanism limits the usage of the browser cache as the whole JS code is invalidated for any UMC related package update. We currently plan to use e-tags or last-modified header such that the browser will not need to download large files. max-age header seems to be less appropriate, as this might lead to situations where files from different versions could be loaded. ATM, UMC loads JavaScript files of all modules when logging in. Then there seem the following problematic use cases when relying on HTTP headers only: * JS module and python could differ in their package version if an update happens during being logged in. This might be problematic if JS and python modules ship API changes. This should not happen during Errata updates, and this should be fairly rare during major/minor updates. * We would need a different mechanism to detect a new module or UMC frontend version. See also: > https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching
(In reply to Alexander Kläser from comment #0) > [...] > See also: > > https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching "The combination of ETag, Cache-Control, and unique URLs [i.e., URLs containing a version string] allows us to deliver the best of all worlds: long-lived expiry times, control over where the response can be cached, and on-demand updates." Updated proposal: * A web application provides a template HTML file (e.g., index.uhtml) which may include references to package versions in the form %{univention-web-js:version} * A call to, e.g., /univention/management will be redirected to the UMC web server which will look for a index.uthml file and replace %{...} occurrences For the templating, we could simply use: ----------- 8< ----------- > class MyTemplate(Template): > idpattern = '[_a-z][_a-z0-9:-]*' > delimiter = '%' > > d = { 'univention-web-js:version': 'v4.1-2' } > t = MyTemplate('... <link rel="stylesheet" href="/univention/js%{univention-web-js:version}/js/dijit/themes/univention/univention.css" type="text/css"/> ...') > t.substitute(d) ----------- 8< -----------
univention-management-console r 72627 | v 9.0.12-15A~4.2.0.201609151534 Bug #38824: * mod_rewrite dijit/themes/umc/ request if they are not found in new location * removed mod_rewrite for hashed javascript directories r 72636 | v 9.0.12-16A~4.2.0.201609151819 Bug #38824: * removed mod_rewrite rule for non existing files under dijit/themes/umc/
I just came across a really nice feature in dojo which could help us manage the caching more appropriately. If dojoConfig.cacheBust = "XYZ" is defined, "?XYZ" is appended as query string to each loaded JavaScript module file. We could use, e.g., the package version of univention-web at this point. When dynamically requiring a UMC module (given the result of get/modules/list), we may add a config object object containing cacheBust again to the call of require(), i.e.: require({ cacheBust:'ZYX' }, [ 'umc/modules/foo' ], function(foo) { // callback function }); Here we could use the package version of the UMC module.
The cacheBust way has been implemented: All univention-web requests uses cacheBust=?prevent-cache=$version-of-univention-web package. All UMC modules are loaded with cacheBust=?version=$package-version Please note that most caches will not cache URLs at all which have a query string. Please note that there is a race condition in dojo as we load all modules asynchronously, the cacheBust is a singleton which causes that modules are loaded with the version string of another module. → revert?
The cacheBust also has the effect, that require.toUrl() returns a URI with query-string. We have various cases where the URL is incorrectly joined leading to some icons and images not being shown. I reverted (part of) the change: univention-web (1.0.28-11): r77085 | Bug #42265: revert adding cacheBust univention-management-console (9.0.42-1): r77085 | Bug #42265: revert adding cacheBust
We decided for the following: Set explicit HTTP caching header to all our static resources. This causes that browser caches don't use a heuristic for how long they cache our content. * For images we set a default of 1 month * application/json has a default of 0 and expires immediately * application/javascript, text/html and text/css files have a default time of one week For some explicit resources we set 'Cache-Control: must-revalidate'. Pressing F5 in the browser causes the necessary files to be reloaded. If we get feedback about problems we will reimplement some versioning as part of the URL. univention-web (1.0.39-1): r77755 | Bug #42265: enhance caching directives r77713 | Bug #42265: Add default Expires and Cache-Control: max-age values r77662 | Bug #42265: forces caches to revalidate meta.json r77085 | Bug #42265: revert adding cacheBust r76931 | Bug #42265: add cacheBust to dojoConfig univention-management-console (9.0.64-1): r77085 | Bug #42265: revert adding cacheBust r76951 | Bug #42265: fix FTBFS r76949 | Bug #42265: add the version number of UMC modules as querystring (dojoConfig.cacheBust) to prevent wrong file to be cached.
There seem to be problems. Maybe we shouldn't set Expires information for Javascript files and force a must-revalidate instead.
We agreed to cache JS files for 1 day. Apart from that, I am wondering whether "ExpiresByType text/css M604800" is ok. Let's say we have an image which is 2 months old. By this rule, we will force everybody not to cache the image anymore.
We also need to cache woff font files.
Created attachment 8612 [details] Proposed patch Attached a patch with some proposed changes.
Created attachment 8613 [details] Proposed patch oops... wrong patch
Thanks for the patch: r78148 | Bug #42265: adjust caching time, cache also fonts
univention-web (1.0.42-1): r78413 | Bug #42265: make caching for access time instead of last-modified time
(In reply to Florian Best from comment #13) > univention-web (1.0.42-1): > r78413 | Bug #42265: make caching for access time instead of last-modified > time With the latest changes Expires header are set correctly per file type. Tested with Chromium and Firefox.
UCS 4.2 has been released: https://docs.software-univention.de/release-notes-4.2-0-en.html https://docs.software-univention.de/release-notes-4.2-0-de.html If this error occurs again, please use "Clone This Bug".