Univention Bugzilla – Bug 36473
logrotate does not trigger applications to reopen their logfiles
Last modified: 2015-05-28 17:54:24 CEST
logrotate does not trigger the running applications to reopen their logfiles. Thus they are still logging into already deleted files and the log entries are lost. root@master:~# lsof | grep .log.1 apache2 1445 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 1629 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 5946 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 5947 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 6637 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) squid3 8153 root 3u REG 253,0 18665 399352 /var/log/squid3/cache.log.1 (deleted) squid3 8156 proxy 3u REG 253,0 18665 399352 /var/log/squid3/cache.log.1 (deleted) univentio 11056 root 4u REG 253,0 37070 400458 /var/log/univention/management-console-web-server.log.1 (deleted) apache2 11104 root 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 18150 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 18157 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 20148 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 23107 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted) apache2 31323 www-data 3u REG 253,0 1069 400448 /var/log/univention/management-console-module-lib.log.1 (deleted)
There are 3 daemons which may be still affected: * univention-directory-manager (univention-cli-server with very high directory/manager/cmd/timeout) * univention-ad-connector (ucs-ad-connector.cpp is a daemon process which uses a rotated logfile) * univention-log-collector-client (log-collector-server.py is a daemon process) I'll investigate those 3 a little bit more. Here's an overview of all other logfiles which are not affected afaics: univention-base-files: /var/log/btmp and /var/log/wtmp: → no daemon process using it rsyslog → "invoke-rc.d rsyslog rotate > /dev/null" univention-heimdal: /var/log/heimdal-kdc.log /var/log/heimdal-database.log univention-maintenance: /var/log/univention/system-stats.log → no daemon, script: univention-system-stats univention-server: /var/log/univention/server_password_change.log → no daemon, script: server_password_change univention-ssl: /var/log/univention/ssl-sync.log → used by a cronjob univention-system-setup: /var/log/univention/setup.log → no daemon, script during applying system setup changes univention-updater: /var/log/univention/{errata-updates,repository,actualise,updater}.log → no daemon, univention-updater/python scripts and postup/preup and UMC updater module univention-spamassassin: /var/log/univention/spamassassin-learn.log → used by a cronjob univention-directory-listener: /var/log/univention/listener.log → test -x /usr/bin/sv && test -e /etc/runit/univention/univention-directory-listener && sv term univention-directory-listener || true univention-directory-manager: sync-memberuid → no daemon, script: univention-sync-memberuid check_group_recursion → no daemon, script: check_group_recursion directory-manager-cmd → used by the follwoing modules: * univention-cli-server → daemon, kills itself after a timeout of 300 seonds by default. The timeout can be configured up to a runtime of 24855 days. Should we fix this here? * modules/univention/admincli/adduser.py → belongs to univention-cli-server * modules/univention/admincli/admin.py → belongs to univention-cli-server * modules/univention/admincli/passwd.py → belongs to univention-cli-server * univention-dnsedit → no daemon, only a script univention-directory-notifier: /var/log/univention/notifier.log → test -x /usr/bin/sv && test -e /etc/runit/univention/univention-directory-notifier && sv term univention-directory-notifier || true univention-directory-policy: ldap-policy → only used by init script which calls run-parts directly piping to the logfile univention-directory-replication: ldap-replication-resync → no daemon, script: univention-directory-replication-resync univention-directory-reports: directory-reports → only during report creation univention-join check_join_status → script: check_join_status.sh server-join → script: univention-server-join UMC was already fixed by Bug #37316 univention-directory-logger: directory-logger → no daemon, listener module "directory_logger.py" univention-ad-connector: connector-tracebacks.log → I could not find any usage ad-connector-certificate.log → UCR template: ad-certificate.py connector-status.log → I could not find any usage connector.log → DAEMON: "ucs-ad-connector.cpp" univention-dhcp: dhcp.log → test -x /usr/bin/sv && test -e /etc/runit/univention/univention-dhcp && sv term univention-dhcp || true univention-log-collector-client: log-collector-client.log → only a client script log-collector-server.log → DAEMON: "log-collector-server.py" univention-pkgdb: pkgdb.log → no daemon, script/cronjob which triggers the execution univention-printserver: printserver-start.log → I could not find any usage univention-s4-connector: *.s4.log → invoke-rc.d univention-s4-connector crestart > /dev/null univention-samba: log.nmbd → kill -HUP `cat /var/run/samba/nmbd.pid` log.smbd → invoke-rc.d --quiet samba reload > /dev/null /var/log/samba/log.samba → /usr/bin/killall -HUP /usr/sbin/samba winbindd →/usr/bin/killall -HUP /usr/sbin/samba univention-virtual-machine-manager-daemon: /var/log/univention/virtual-machine-manager-daemon.log → sv hup univention-virtual-machine-manager-daemon and kill -HUP "$(cat /var/run/uvmmd.pid)"
(In reply to Florian Best from comment #1) > * univention-ad-connector (ucs-ad-connector.cpp is a daemon process which > uses a rotated logfile) → this is only a application running on windows. No need to fix anything. > * univention-log-collector-client (log-collector-server.py is a daemon > process) → Who uses this? I can add a postrotate command which does "invoke-rc.d univention-log-collector-server restart" but this would interrupt connections if there are any. It does not use univention-debug. I could also fix it with SIGHUP but is this important to fix now? Ah, and forgot these: > univention-heimdal: > /var/log/heimdal-kdc.log → used by /usr/lib/heimdal-servers/kdc, referenced in /etc/heimdal-kdc/kdc.conf > /var/log/heimdal-database.log → referenced in /etc/heimdal-kdc/kdc.conf and etc/krb5.conf
heimdal → Bug #37594 univention-cli-server → WONTFIX, if this occur set the livetime down to max. 1 day log-collector-server.py → WONTFIX, it is not often used
After two forced log rotations, the logfiles access.log and cache.log seem to be reopened by squid3 and active: root@master:~# date Mi 11. Mär 11:38:12 CET 2015 root@master:~# logrotate -vf /etc/logrotate.conf root@master:~# logrotate -vf /etc/logrotate.conf root@master:~# cat /var/log/squid3/cache.log 2015/03/11 11:38:52| storeDirWriteCleanLogs: Starting... 2015/03/11 11:38:52| Finished. Wrote 0 entries. 2015/03/11 11:38:52| Took 0.00 seconds ( 0.00 entries/sec). 2015/03/11 11:38:52| logfileRotate: /var/log/squid3/access.log root@master:~# lsof | grep /var/log/squid3 squid3 1496 root 3u REG 253,0 8340 2238097 /var/log/squid3/cache.log.1 (deleted) squid3 1500 proxy 3u REG 253,0 8340 2238097 /var/log/squid3/cache.log.1 (deleted) squid3 1500 proxy 5u REG 253,0 234 2238099 /var/log/squid3/cache.log squid3 1500 proxy 9w REG 253,0 0 2238103 /var/log/squid3/access.log root@master:~# Looks like the parent squid process and the child keep an outdated filedescriptor open. Since the open fds do not get more, I think we can ignore that. ...but... REOPEN: The logfile log.samba is not reopened by samba4. root@master:/etc/logrotate.d# lsof | grep /var/log/samba/log.samba.1 samba 13288 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13288 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13289 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13289 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13290 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13290 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13291 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13291 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13292 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13292 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) smbd 13293 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13294 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13294 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13295 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13295 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13296 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13296 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13297 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13297 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13298 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13298 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13299 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13299 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) smbd 13309 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) root@master:/etc/logrotate.d# /usr/bin/killall -HUP -i /usr/sbin/samba Signal samba(13275) senden? (y/N) y Signal samba(13288) senden? (y/N) y Signal samba(13289) senden? (y/N) y Signal samba(13290) senden? (y/N) y Signal samba(13291) senden? (y/N) y Signal samba(13292) senden? (y/N) y Signal samba(13294) senden? (y/N) y Signal samba(13295) senden? (y/N) y Signal samba(13296) senden? (y/N) y Signal samba(13297) senden? (y/N) y Signal samba(13298) senden? (y/N) y Signal samba(13299) senden? (y/N) y root@master:/etc/logrotate.d# lsof | grep /var/log/samba/log.samba.1 samba 13288 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13288 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13289 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13289 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13290 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13290 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13291 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13291 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13292 root 2w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) samba 13292 root 3w REG 253,0 4255 2108662 /var/log/samba/log.samba.1 (deleted) [...]
Samba4 → Bug #38563
(In reply to Florian Best from comment #5) > Samba4 → Bug #38563 OK