Bug 45960

Summary:	Listener hangs due to saml restart in UCS 4.3 setup
Product:	UCS	Reporter:	Stefan Gohmann <gohmann>
Component:	SAML	Assignee:	Stefan Gohmann <gohmann>
Status:	CLOSED FIXED	QA Contact:	Jürn Brodersen <brodersen>
Severity:	normal
Priority:	P5	CC:	best, brodersen
Version:	UCS 4.3	Keywords:	interim-1
Target Milestone:	UCS 4.3
Hardware:	Other
OS:	Linux
See Also:	https://forge.univention.org/bugzilla/show_bug.cgi?id=43486
What kind of report is it?:	Development Internal	What type of bug is this?:	---
Who will be affected by this bug?:	---	How will those affected feel about the bug?:	---
User Pain:		Enterprise Customer affected?:
School Customer affected?:		ISV affected?:
Waiting Support:		Flags outvoted (downgraded) after PO Review:
Ticket number:		Bug group (optional):
Max CVSS v3 score:
Bug Depends on:
Bug Blocks:	46212
Attachments:	autotest-091-master-s4.log.bz2 journalctl.lst syslog daemon.log system-stats.log

Description Stefan Gohmann

2017-12-28 07:25:49 CET

The listener gets stuck during UCS 4.3 Amazon EC2 setup:

root     16487  0.0  0.0   4060   616 ?        Ss   01:08   0:00  \_ runsv univention-directory-listener
root     23083 12.6  1.4 2716512 56796 ?       S    01:21   0:05  |   \_ /usr/sbin/univention-directory-listener -F -d 4 -b dc=AutoTest091,dc=local -m /usr/lib/univention-directory-listener/system -c /var/lib/un
root     23138  0.0  0.0   4288  1596 ?        S    01:21   0:00  |       \_ /bin/sh /usr/sbin/invoke-rc.d univention-saml restart
root     23147  0.0  0.0  37948  2152 ?        S    01:21   0:00  |           \_ systemctl restart univention-saml.service


It looks like the restart doesn't return:
root     23100  0.0  0.0  13028  3392 ?        Ss   01:21   0:00 /bin/bash /etc/init.d/univention-saml stop
root     23121  0.0  0.0   4288  1628 ?        S    01:21   0:00  \_ /bin/sh -e /etc/init.d/stunnel4 stop
root     23127  0.0  0.0  41128  2200 ?        S    01:21   0:00      \_ /bin/systemctl --no-pager stop stunnel4.service


I've changed the listener debug to 4 and set system stats to run every minute.

Comment 1 Stefan Gohmann

2017-12-28 07:29:08 CET

Created attachment 9331 [details]
autotest-091-master-s4.log.bz2

Comment 2 Stefan Gohmann

2017-12-28 07:29:24 CET

Created attachment 9332 [details]
journalctl.lst

Comment 3 Stefan Gohmann

2017-12-28 07:29:43 CET

Created attachment 9333 [details]
syslog

Comment 4 Stefan Gohmann

2017-12-28 07:30:01 CET

Created attachment 9334 [details]
daemon.log

Comment 5 Stefan Gohmann

2017-12-28 07:30:19 CET

Created attachment 9335 [details]
system-stats.log

Comment 6 Stefan Gohmann

2017-12-28 07:47:57 CET

It is reproducible in this way:

root@master093:~# /etc/init.d/memcached start univention_saml
[ ok ] Starting memcached (via systemctl): memcached.service.
root@master093:~# /etc/init.d/stunnel4 start
[ ok ] Starting stunnel4 (via systemctl): stunnel4.service.
root@master093:~# /etc/init.d/memcached stop univention_saml
[ ok ] Stopping memcached (via systemctl): memcached.service.
root@master093:~# /etc/init.d/stunnel4 stop
[ ok ] Stopping stunnel4 (via systemctl): stunnel4.service.
root@master093:~# /etc/init.d/memcached start univention_saml
[ ok ] Starting memcached (via systemctl): memcached.service.
root@master093:~# /etc/init.d/stunnel4 start
[ ok ] Starting stunnel4 (via systemctl): stunnel4.service.
root@master093:~# /etc/init.d/univention-saml stop
[....] Stopping univention-saml (via systemctl): univention-saml.service

→ hangs

Comment 7 Stefan Gohmann

2017-12-28 15:12:20 CET

Looks like Bug #43486 is one problem.

Comment 8 Stefan Gohmann

2017-12-29 15:01:26 CET

*** Bug 43486 has been marked as a duplicate of this bug. ***

Comment 9 Stefan Gohmann

2017-12-29 15:17:44 CET

I've added a systemd script for univention_saml memcache:

https://git.knut.univention.de/univention/ucs/commit/f5b4726b78e725cab2d2ab404a5be2cf390baecf

The stunnel init script now also creates the piddir /var/run/univention-saml. SVN r17918 - r17923.

Changelog:
https://git.knut.univention.de/univention/ucs/commit/fc07c302286df558da7d291e8fac2e906f4becd8

Comment 10 Jürn Brodersen

2018-01-05 12:35:06 CET

Is there a reason you didn't delete the init file?
The service file replaces the init file. If possible please delete it or replace it by a file that just calls:
'''
# Inject systemd
. /lib/lsb/init-functions
'''
to make that more clear :)
Afaik systemd stops processing an init file if a service file under the same name exists and starts the service instead.

Comment 11 Stefan Gohmann

2018-01-14 19:28:19 CET

(In reply to Jürn Brodersen from comment #10)
> Is there a reason you didn't delete the init file?

Yes, I saw code like 'test -x /etc/init.d/unviention-saml && service restart univention-saml'

> The service file replaces the init file. If possible please delete it or
> replace it by a file that just calls:
> '''
> # Inject systemd
> . /lib/lsb/init-functions
> '''
> to make that more clear :)
> Afaik systemd stops processing an init file if a service file under the same
> name exists and starts the service instead.

Yes, I removed the code from the univention saml init file:

https://git.knut.univention.de/univention/ucs/commit/5419472c08fa6703e2d97138f4e3a33782d6204a

Comment 12 Stefan Gohmann

2018-01-14 19:37:20 CET

*** Bug 45925 has been marked as a duplicate of this bug. ***

Comment 13 Jürn Brodersen

2018-01-17 17:17:12 CET

Looks good now :)

What I tested:
Installation -> OK
systemctl status univention-saml -> OK
systemctl status univention-directory-listener -> OK

-> Verified

Comment 14 Stefan Gohmann

2018-03-14 14:38:36 CET

UCS 4.3 has been released:
 https://docs.software-univention.de/release-notes-4.3-0-en.html
 https://docs.software-univention.de/release-notes-4.3-0-de.html

If this error occurs again, please use "Clone This Bug".