Univention Bugzilla – Bug 37159
logrotate does not trigger UMC components to reopen their logfiles
Last modified: 2015-05-19 13:40:36 CEST
In case there is a crash with UMC, this problem leads to empty UMC log files: * logrotate renames the log files → UMC server process still holds the file handle to the moved files * after a day, logrotate compresses the log file → output into the old file handles is not written anymore to hard disk In addition to Bug 36473, UMC server would need to handle the corresponding signal emitted by logrotate. +++ This bug was initially created as a clone of Bug #36473 +++ logrotate does not trigger the running applications to reopen their logfiles. Thus they are still logging into already deleted files and the log entries are lost. root@master:~# lsof | grep .log.1 apache2 1445 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 1629 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 5946 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 5947 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 6637 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) squid3 8153 root 3u REG 253,0 18665 399352 /var/log/squid3/cache.log.1 (deleted) squid3 8156 proxy 3u REG 253,0 18665 399352 /var/log/squid3/cache.log.1 (deleted) univentio 11056 root 4u REG 253,0 37070 400458 /var/log/univention/management-console-web-server.log.1 (deleted) apache2 11104 root 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 18150 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 18157 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 20148 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 23107 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 31323 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted)
This very problem was also seen at Ticket #2014110221000053 and the corresponding USI. Somehow the UMC server was not available during exam mode but no further analysis is possible, because UMC's has not been updated for ~4 days.
Logrotate now sends a SIGHUP (1) to the UMC-web-server and UMC-server process. UMC-server and web-server triggers univention.debug.reload(). A segfault there has been fixed. Fix: svn r56747 r56746 r56758 r56761 r56763 2014-12-12-univention-debug.yaml (3.2-3 & 3.2-4) 2014-12-12-univention-management-console-frontend.yaml (only 3.2-3) 2014-12-12-univention-management-console.yaml (3.2-3 & 3.2-4)
*** Bug 36603 has been marked as a duplicate of this bug. ***
QA: the following command helps to manually trigger a logrotate: logrotate -f /etc/logrotate.d/univention-management-console
Some notes for the QA: It has to be tested what happens when logrotate triggers the rotation of all UMC logfiles. After rotation the logfiles /var/log/univention/management-console-{server,web-server,module-computerroom,module-udm}.log should be existant and contain the content of all log messages which are done after rotation. The logrotation sends a SIGHUP signal to the umc-webserver umc-server and all running module processes. Please check if all processes still run after a logrotation. Especially the UCS@school computerroom module - as it also uses QT as notifier implementation. And one normal module like UDM. I removed the cross dependency between the packages. Test what happens if you only upgrade univention-management-console-web-server before the rest of the packages. A invoke-rc.d univention-management-console-web-server restart should still work and a umc login should be possible.
REOPEN: The logrotate script is not executable and does not exit with 0 (so the last command exitcode is used which is 1 when no UMC module processes exists).
A test case has been added to ucs-test for UCS 4.0: svn r56851
(In reply to Florian Best from comment #6) > REOPEN: The logrotate script is not executable and does not exit with 0 (so > the last command exitcode is used which is 1 when no UMC module processes > exists). This has been fixed in svn r56852 r56853 r56854.
OK - Adapted yaml 56900 OK - logs after logrotate OK - process alive OK - changelog
http://errata.univention.de/ucs/3.2/251.html
http://errata.univention.de/ucs/3.2/250.html
http://errata.univention.de/ucs/3.2/249.html
http://errata.univention.de/ucs/3.2/256.html