Bug 44052 - UMC server is not executable
UMC server is not executable
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.2
Assigned To: Janek Walkenhorst
Stefan Gohmann
: interim-4
: 44029 44049 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-03-23 17:16 CET by Sönke Schwardt-Krummrich
Modified: 2017-04-04 18:28 CEST (History)
6 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 5: Will affect all installed domains
How will those affected feel about the bug?: 4: A User would return the product
User Pain: 0.686
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sönke Schwardt-Krummrich univentionstaff 2017-03-23 17:16:48 CET
I tried to install UCS@school on a UCS 4.2-0 from test app center.
At about 46% the UMC server was shut down and didn't come up again.

Mär 18 03:04:02 master systemd[1]: Stopping LSB: Univention Management Console Server...
Mär 18 03:04:03 master systemd[1]: univention-management-console-server.service: main process exited, code=exited, status=1/FAILURE
Mär 18 03:04:03 master univention-management-console-server[16222]: Stopping Univention Management Console Server: univention-management-console-server.
Mär 18 03:04:03 master systemd[1]: Stopped LSB: Univention Management Console Server.
Mär 18 03:04:03 master systemd[1]: Unit univention-management-console-server.service entered failed state.
Mär 18 03:04:03 master systemd[1]: Reloading.
Mär 18 03:04:03 master systemd[1]: [/lib/systemd/system/docker.service:22] Unknown lvalue 'TasksMax' in section 'Service'
Mär 18 03:04:03 master systemd[1]: [/lib/systemd/system/docker.service:25] Unknown lvalue 'Delegate' in section 'Service'
Mär 18 03:04:03 master systemd[1]: Started ACPI event daemon.
Mär 18 03:04:03 master systemd[1]: Listening on ACPID Listen Socket.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/mapper/vg_ucs-swap_1.
Mär 18 03:04:03 master systemd[1]: Found device /dev/mapper/vg_ucs-swap_1.
Mär 18 03:04:03 master systemd[1]: Mounted /.
Mär 18 03:04:03 master systemd[1]: Found device /sys/devices/virtual/block/dm-1.
Mär 18 03:04:03 master systemd[1]: Found device /dev/dm-1.
Mär 18 03:04:03 master systemd[1]: Found device /dev/disk/by-id/dm-name-vg_ucs-swap_1.
Mär 18 03:04:03 master systemd[1]: Found device /dev/disk/by-id/dm-uuid-LVM-Y7NYUEhlL9RnIKB6s2GJo65icG0J33CKGGJheXgfoR3e3VpOfirN6ZZhFJvmVuVi.
Mär 18 03:04:03 master systemd[1]: Found device /dev/disk/by-uuid/8ffc132b-017f-4ac1-877d-4b790d99ac47.
Mär 18 03:04:03 master systemd[1]: Found device /dev/vg_ucs/swap_1.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/dm-1.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/disk/by-id/dm-name-vg_ucs-swap_1.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/disk/by-id/dm-uuid-LVM-Y7NYUEhlL9RnIKB6s2GJo65icG0J33CKGGJheXgfoR3e3VpOfirN6ZZhFJvmVuVi.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/disk/by-uuid/8ffc132b-017f-4ac1-877d-4b790d99ac47.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/vg_ucs/swap_1.
Mär 18 03:04:03 master systemd[1]: Unit univention-management-console-server.service cannot be reloaded because it is inactive.
Mär 18 03:04:03 master systemd[1]: Reloading.
Mär 18 03:04:03 master systemd[1]: [/lib/systemd/system/docker.service:22] Unknown lvalue 'TasksMax' in section 'Service'
Mär 18 03:04:03 master systemd[1]: [/lib/systemd/system/docker.service:25] Unknown lvalue 'Delegate' in section 'Service'
Mär 18 03:04:03 master systemd[1]: Started ACPI event daemon.
Mär 18 03:04:03 master systemd[1]: Listening on ACPID Listen Socket.
Mär 18 03:04:03 master systemd[1]: Activated swap /dev/mapper/vg_ucs-swap_1.
Mär 18 03:04:03 master systemd[1]: Found device /dev/mapper/vg_ucs-swap_1.
[…]


root@master:~# systemctl status univention-management-console-server.service
● univention-management-console-server.service - LSB: Univention Management Console Server
   Loaded: loaded (/etc/init.d/univention-management-console-server)
   Active: failed (Result: timeout) since Sa 2017-03-18 04:40:43 CET; 6min ago
 Main PID: 30941 (code=exited, status=1/FAILURE)

Mär 18 04:35:42 master systemd[1]: PID file /var/run/umc-server.pid not readable (yet?) after start.
Mär 18 04:40:43 master systemd[1]: univention-management-console-server.service start operation timed out. Terminating.
Mär 18 04:40:43 master systemd[1]: Failed to start LSB: Univention Management Console Server.
Mär 18 04:40:43 master systemd[1]: Unit univention-management-console-server.service entered failed state.
Mär 18 04:45:46 master slaptest[23777]: Loaded metadata from "/usr/share/univention-management-console/saml/idp/ucs-sso.nstx.local.xml"
Mär 18 04:45:46 master slaptest[23777]: DIGEST-MD5 common mech free
Mär 18 04:45:46 master slapindex[23779]: Loaded metadata from "/usr/share/univention-management-console/saml/idp/ucs-sso.nstx.local.xml"
Mär 18 04:45:56 master slapindex[23779]: DIGEST-MD5 common mech free
Mär 18 04:46:09 master gpasswd[24332]: user master$ added by root to group lp
Mär 18 04:47:08 master systemd[1]: Unit univention-management-console-server.service cannot be reloaded because it is inactive.
Comment 1 Sönke Schwardt-Krummrich univentionstaff 2017-03-23 17:18:16 CET
Maybe related to/same issue as #44049
Comment 2 Sönke Schwardt-Krummrich univentionstaff 2017-03-23 17:27:11 CET
/var/run/umc-server.pid does not exist

root@master:~# ls -la /usr/sbin/univention-management-console-server 
-rw-r--r-- 1 root root 8144 Mär 23  2017 /usr/sbin/univention-management-console-server

→ wrong permissions for the executable

# chmod +x /usr/sbin/univention-management-console-server 
# systemctl status univention-management-console-server.service 
● univention-management-console-server.service - LSB: Univention Management Console Server
   Loaded: loaded (/etc/init.d/univention-management-console-server)
   Active: active (running) since Sa 2017-03-18 05:06:22 CET; 3s ago
  Process: 26275 ExecStart=/etc/init.d/univention-management-console-server start (code=exited, status=0/SUCCESS)
 Main PID: 26283 (univention-mana)
   CGroup: /system.slice/univention-management-console-server.service
           └─26283 /usr/bin/python2.7 /usr/sbin/univention-management-console-server start

Mär 18 05:06:22 master univention-management-console-server[26275]: Starting Univention Management Console Server: univention-management-console-server.
Mär 18 05:06:22 master systemd[1]: PID file /var/run/umc-server.pid not readable (yet?) after start.
Mär 18 05:06:22 master systemd[1]: Started LSB: Univention Management Console Server.
Comment 3 Florian Best univentionstaff 2017-03-23 18:51:44 CET
When updating via UMC we are removing the executable bits on purpose.
Comment 4 Sönke Schwardt-Krummrich univentionstaff 2017-03-23 21:00:48 CET
(In reply to Florian Best from comment #3)
> When updating via UMC we are removing the executable bits on purpose.

systemd does not like this. Instead UMC server is stopped via systemd and cannot be restarted.
Comment 5 Alexander Kläser univentionstaff 2017-03-23 21:42:33 CET
(In reply to Sönke Schwardt-Krummrich from comment #4)
> (In reply to Florian Best from comment #3)
> > When updating via UMC we are removing the executable bits on purpose.
> 
> systemd does not like this. Instead UMC server is stopped via systemd and
> cannot be restarted.

Oh, then we have a problem as this feature is used a various places...

> $ rgrep -l disable-apache2-umc
> base/univention-lib/python/package_manager.py
> base/univention-system-setup/umc/python/setup/util.py
> base/univention-system-setup/usr/lib/univention-system-setup/cleanup-pre.d/99_restart_umc
> base/univention-system-setup/usr/lib/univention-system-setup/scripts/setup-join.sh
> base/univention-updater/debian/changelog
> base/univention-updater/debian/rules
> base/univention-updater/debian/univention-updater.install
> base/univention-updater/umc/python/updater/__init__.py
> management/univention-join/umc/python/join/__init__.py
> management/univention-management-console-module-adtakeover/umc/python/adtakeover/takeover.py
> management/univention-management-console-module-lib/umc/python/lib/server.py
Comment 6 Stefan Gohmann univentionstaff 2017-03-24 05:52:25 CET
I've started a UCS 4.2 package update (installed yesterday) via UMC and the update get stuck:

root@master551:~# tail /var/log/univention/updater.log 
Entpacken von docker.io (1.12.6-0ubuntu1~16.04.1A~4.2.0.201703231219) über (1.12.6-0ubuntu1~16.04.1A~4.2.0.201703081708) ...
Vorbereitung zum Entpacken von .../univention-web-js_1.0.40-11A~4.2.0.201703240032_all.deb ...
Entpacken von univention-web-js (1.0.40-11A~4.2.0.201703240032) über (1.0.40-4A~4.2.0.201703231016) ...
Vorbereitung zum Entpacken von .../univention-web-style_1.0.40-11A~4.2.0.201703240032_all.deb ...
Entpacken von univention-web-style (1.0.40-11A~4.2.0.201703240032) über (1.0.40-4A~4.2.0.201703231016) ...
Vorbereitung zum Entpacken von .../univention-management-console-login_9.0.72-4A~4.2.0.201703231939_all.deb ...
Entpacken von univention-management-console-login (9.0.72-4A~4.2.0.201703231939) über (9.0.70-1A~4.2.0.201703221926) ...
Vorbereitung zum Entpacken von .../univention-management-console-frontend_9.0.72-4A~4.2.0.201703231939_all.deb ...
Entpacken von univention-management-console-frontend (9.0.72-4A~4.2.0.201703231939) über (9.0.70-1A~4.2.0.201703221926) ...
Vorbereitung zum Entpacken von .../univention-management-console-web-server_9.0.72-4A~4.2.0.201703231939_all.deb ...
root@master551:~# 


root     17195  0.0  0.0   4336   756 pts/0    SN+  05:43   0:00                      \_ /bin/sh /var/lib/dpkg/info/univention-management-console-web-server.prerm upgrade 9.0.72-4A~4.2.0.201703231939
root     17198  0.0  0.0   4336  1408 pts/0    SN+  05:43   0:00                          \_ /bin/sh /usr/sbin/invoke-rc.d univention-management-console-web-server stop
root     17228  0.0  0.1  23956  2640 pts/0    SN+  05:43   0:00                              \_ systemctl stop univention-management-console-web-server.service

root@master551:~# ls -la /usr/sbin/univention-management-console-server /usr/sbin/univention-management-console-web-server 
-rw-r--r-- 1 root root  8144 Mär 23 19:43 /usr/sbin/univention-management-console-server
-rw-r--r-- 1 root root 52684 Mär 23 17:00 /usr/sbin/univention-management-console-web-server

Maybe we could simply add a ExecStartPre= to the systemd service.

       ExecStartPre=, ExecStartPost=
           Additional commands that are executed before or after the command in ExecStart=, respectively. Syntax is the same as for ExecStart=, except that multiple command lines are
           allowed and the commands are executed one after the other, serially.

           If any of those commands (not prefixed with "-") fail, the rest are not executed and the unit is considered failed.
Comment 7 Stefan Gohmann univentionstaff 2017-03-24 05:56:03 CET
*** Bug 44029 has been marked as a duplicate of this bug. ***
Comment 8 Stefan Gohmann univentionstaff 2017-03-24 05:57:23 CET
*** Bug 44049 has been marked as a duplicate of this bug. ***
Comment 9 Stefan Gohmann univentionstaff 2017-03-24 07:16:53 CET
(In reply to Stefan Gohmann from comment #6)
> Maybe we could simply add a ExecStartPre= to the systemd service.

And of course something which blocks the stopping process if the executable flag is missing. ExecStop= can be added multiple times.
Comment 10 Janek Walkenhorst univentionstaff 2017-03-24 14:26:54 CET
univention-management-console (9.0.73-3) unstable; urgency=medium

  * Bug #44052: repair "removing the executable bit" for systemd
Comment 11 Florian Best univentionstaff 2017-03-24 17:05:57 CET
I think we need to do the same for Apache at least until the restart of the apache2.2 from UCS 4.1. Currently it shows:

Restarting web server: apache2 failed!
There are processes named 'apache2' running which do not match your pid file which are left untouched in the name of safety, Please review the situation by hand. ... (warning).

This happens probably because the new config file of the service is active, which contains pidfile=….
Comment 12 Florian Best univentionstaff 2017-03-24 17:08:28 CET
The PID is btw correct :/
# pgrep -flF /var/run/apache2.pid
15372 apache2
Comment 13 Florian Best univentionstaff 2017-03-24 17:30:39 CET
I fixed that in Bug #43520#c12
Comment 14 Stefan Gohmann univentionstaff 2017-03-26 18:54:34 CEST
OK, it works now. The pidfile is no longer removed by systemd.
Comment 15 Stefan Gohmann univentionstaff 2017-04-04 18:28:16 CEST
UCS 4.2 has been released:
 https://docs.software-univention.de/release-notes-4.2-0-en.html
 https://docs.software-univention.de/release-notes-4.2-0-de.html

If this error occurs again, please use "Clone This Bug".