Bug 45343 - UMC system diagnostic module hangs for a long time (4.3)
UMC system diagnostic module hangs for a long time (4.3)
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - System diagnostic
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.3-0-errata
Assigned To: Arvid Requate
Felix Botner
:
Depends on: 37032
Blocks: 48002
  Show dependency treegraph
 
Reported: 2017-09-07 16:40 CEST by Arvid Requate
Modified: 2018-10-16 14:01 CEST (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.091
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
Screenshot_20170907_162957.png (97.76 KB, image/png)
2017-09-07 16:40 CEST, Arvid Requate
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arvid Requate univentionstaff 2017-09-07 16:40:51 CEST
Created attachment 9181 [details]
Screenshot_20170907_162957.png

Soemtimes the UMC system diagnostic module hangs for a long time, several minutes (or infinitely?). See attached screenshot. All tests seem to run but finally it hangs at 96% with the last message "Diagnosis of "Gateway is not reachable" was successful". It is not really clear if the module is still doing something.


My environment consisted of a UCS 4.2-2 Master and Backup, both with Samba/AD and I observed this during QA for Bug #36748. I had killed the KDC on the DC Backup for QA purposes.

In case it hangs, I can instruct the browser to reload the page, and then it works (always?). But when I click on "Run tests again", it hangs again.


We had this before under different circumstances and fixed in in the bug I cloned.


+++ This bug was initially created as a clone of Bug #37032 +++
Comment 1 Arvid Requate univentionstaff 2017-09-07 16:49:36 CEST
While I filed this bug, an error message popped up:

 Notification
An unknown error with status code 502 occurred while connecting to the server, please try again later.

I hit "RUN SYSTEM DIAGNOSTICS" again:

* First time: Works
* Second time: Works
* Third time: Hangs
Comment 2 Erik Damrose univentionstaff 2017-09-08 09:09:14 CEST
Which UCS Version / errata level?
Comment 3 Arvid Requate univentionstaff 2017-09-08 12:02:39 CEST
> My environment consisted of a UCS 4.2-2 Master and Backup, both with Samba/AD
> and I observed this during QA for Bug #36748.

I guess errata level 159
Comment 4 Arvid Requate univentionstaff 2018-03-20 14:07:36 CET
It's diagnostic/plugins/01_ssh_connection.py that sometimes hangs:

https://github.com/paramiko/paramiko/issues/515
https://stackoverflow.com/questions/35266753/paramiko-python-module-hangs-at-stdout-read

This workaround seemed to fix it for me:

8a1ef5f166 | Workaround for Python paramiko hangs
4d9067fe2e | Advisory

Fixing together with Bug 45904 and Bug 45584.
Comment 5 Felix Botner univentionstaff 2018-03-21 13:27:04 CET
OK - ssh check
OK - YAML
Comment 6 Arvid Requate univentionstaff 2018-03-28 13:28:26 CEST
<http://errata.software-univention.de/ucs/4.3/6.html>
Comment 7 Christian Völker univentionstaff 2018-10-16 13:10:12 CEST
Just happened on a customer system running 4.3-2 errata257.

Any workaround or further troubleshooting steps?

Not sure if it is helpful but pstree -achls shows:
===================================================
systemd splash
├─acpid
├─agetty --noclear tty1 linux
├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ ├─apache2 -k start
│ └─apache2 -k start
├─atd -f
├─blkmapd
├─cron -f
├─dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
├─dockerd -H fd:// --storage-driver=overlay --live-restore --bip=172.17.42.1/16
│ ├─docker-containe -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --shim docker-containerd-shim --runtime docker-runc
│ │ ├─{docker-containe}
│ │ ├─{docker-containe}
│ │ ├─{docker-containe}
│ │ ├─{docker-containe}
│ │ ├─{docker-containe}
│ │ ├─{docker-containe}
│ │ ├─{docker-containe}
│ │ └─{docker-containe}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ ├─{dockerd}
│ └─{dockerd}
├─inetd
├─irqbalance --foreground
├─lvmetad -f
├─master -w
│ ├─pickup -l -t unix -u
│ ├─qmgr -l -t unix -u
│ └─tlsmgr -l -t unix -u
├─memcached -m 64 -s /var/run/univention-saml/memcached.socket -u samlcgi
│ ├─{memcached}
│ ├─{memcached}
│ ├─{memcached}
│ ├─{memcached}
│ ├─{memcached}
│ └─{memcached}
├─memcached -m 64 -p 11211 -u memcache -l 127.0.0.1
│ ├─{memcached}
│ ├─{memcached}
│ ├─{memcached}
│ ├─{memcached}
│ ├─{memcached}
│ └─{memcached}
├─named -c /etc/bind/named.conf.samba4 -f -d 0
│ ├─{named}
│ ├─{named}
│ ├─{named}
│ └─{named}
├─nmbd -D
│ └─nmbd -D
├─nrpe -c /etc/nagios/nrpe.cfg -f
├─nscd
│ ├─{nscd}
│ ├─{nscd}
│ ├─{nscd}
│ ├─{nscd}
│ ├─{nscd}
│ ├─{nscd}
│ └─{nscd}
├─ntpd -p /var/run/ntpd.pid -g -u 114:123
│ └─{ntpd}
├─python2.7 -W ignore /usr/lib/pymodules/python2.7/univention/s4connector/s4/main.py
├─rpc.gssd
├─rpc.idmapd
├─rpc.mountd --manage-gids --port 32767
├─rpcbind -f -w
├─rsyslogd -n
│ ├─{in:imklog}
│ ├─{in:imuxsock}
│ └─{rs:main Q:Reg}
├─runsvdir -P /etc/service log: ...........................................................................................................................................................................................................................................................................................................................................................................................................
│ ├─runsv univention-directory-notifier
│ │ └─univention-dire -o -d 1 -F
│ └─runsv univention-directory-listener
│ └─univention-dire -F -d 2 -b dc=domain,dc=com -m /usr/lib/univention-directory-listener/system -c /var/lib/univention-directory-listener -ZZ -x -D cn=admin,dc=domain,dc=com -y /etc/ldap.secret
├─samba -D
│ ├─samba -D
│ │ └─samba -D
│ │ └─smbd -D --option=server role check:inhibit=yes --foreground
│ │ ├─cleanupd -D --option=server role check:inhibit=yes --foreground
│ │ ├─lpqd -D --option=server role check:inhibit=yes --foreground
│ │ └─smbd-notifyd -D --option=server role check:inhibit=yes --foreground
│ ├─samba -D
│ ├─samba -D
│ ├─samba -D
│ │ └─samba -D
│ ├─samba -D
│ ├─samba -D
│ ├─samba -D
│ ├─samba -D
│ │ └─samba -D
│ │ └─winbindd -D --option=server role check:inhibit=yes --foreground
│ │ └─winbindd -D --option=server role check:inhibit=yes --foreground
│ ├─samba -D
│ ├─samba -D
│ └─samba -D
├─slapd -h ldapi:/// ldap://:7389/ ldaps://:7636/
│ ├─{slapd}
│ ├─{slapd}
│ ├─{slapd}
│ ├─{slapd}
│ └─{slapd}
├─sshd -D
│ ├─sshd
│ │ └─bash
│ │ └─pstree -alhcs
│ └─sshd
│ └─sftp-server
├─stunnel4 /etc/stunnel/univention_saml.conf
├─systemd-journal
├─systemd-logind
├─systemd-udevd
├─univention-mana /usr/sbin/univention-management-console-server start
│ └─univention-mana /usr/sbin/univention-management-console-module -m diagnostic -s /var/run/univention-management-console/29243-1538126911743.socket -d 2 -l en_US.UTF-8
│ ├─{univention-mana}
│ ├─{univention-mana}
│ └─{univention-mana}
├─univention-mana /usr/sbin/univention-management-console-web-server start
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ ├─{univention-mana}
│ └─{univention-mana}
└─univention-welc /usr/bin/univention-welcome-screen
└─hexdump -n 96 /dev/input/event0 
================================================
Comment 8 Arvid Requate univentionstaff 2018-10-16 13:59:37 CEST
Thus Bug has been fixed with

<http://errata.software-univention.de/ucs/4.3/6.html>

In this case the Bug should be cloned. I've done this: Bug #48002