Bug 39474 - UCS Docker containers not shutting down properly
UCS Docker containers not shutting down properly
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Docker
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1
Assigned To: Arvid Requate
Daniel Tröder
: interim-2
: 38259 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-05 12:36 CEST by Daniel Tröder
Modified: 2017-05-02 11:02 CEST (History)
2 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2015-10-05 12:36:07 CEST
"docker stop <containerid>" is not shutting down containers in
* docker-test.software-univention.de/ucs-master-amd64:4.1-0
* docker-test.software-univention.de/ucs-appbox-amd64:4.1-0

"docker stop <containerid>" does "kill -TERM <containers start process>" which is /sbin/init in the case of UCS. sysv-"init" is build to ignore TERM. So a UCS container does not shutdown.

In the end "docker stop <containerid>" kills the init after <timeout> (defaults to 10s).

The hosts /etc/init.d/docker should be modifies to shut down containers by executing "halt" inside container, and then "docker stop --time=<timeout> <containerid>" in the host.
Comment 1 Daniel Tröder univentionstaff 2015-10-05 16:09:54 CEST
In commit 64228 the docker.io init script was added to the package container/univention-docker.

/usr/lib/univention-docker/univention-docker_lib.sh contains functions that are used in /etc/init.d/docker to handle shutdown and automatic start of containers when the init script is called with start, stop or restart.

Only those containers that were running before stop was called are started automatically.
To start all stopped containers run start_all_stopped_containers after sourcing /usr/lib/univention-docker/univention-docker_lib.sh.
Comment 2 Arvid Requate univentionstaff 2015-10-05 20:02:02 CEST
Ok, works quite well, sometimes I had

======================================================
Stopping Docker:  shutting down containers... init: timeout opening/writing control channel /run/initctl
137
158ce761e952
 , engine....
======================================================

I suggested a couple of adjustments via email.
Comment 3 Daniel Tröder univentionstaff 2015-10-06 10:38:22 CEST
Commit 64242 adds variable quoting, "docker exec" wait-retry and kernel version check.
Comment 4 Arvid Requate univentionstaff 2015-10-07 19:01:56 CEST
Ok, I added another cosmetic and robustness change, now it look fine:

root@master60:~# /etc/init.d/docker restart; /etc/init.d/docker restart
Stopping Docker:  shutting down containers...  8e8a5020b61e fff8bf71a5b5 engine....
Starting Docker: engine...  previously stopped containers...  8e8a5020b61e fff8bf71a5b5.
Stopping Docker:  shutting down containers...  fff8bf71a5b5 8e8a5020b61e engine....
Starting Docker: engine...  previously stopped containers...  8e8a5020b61e fff8bf71a5b5.

I also added a changelog entry, reassigning QA..
Comment 5 Daniel Tröder univentionstaff 2015-10-12 11:53:49 CEST
The docker-app-container-init-script kills the containers in restart() hard with "docker exec "$container_uuid" halt" after 10s, and doesn't wait for a shutdown.
Suggested modification by email.
Comment 6 Arvid Requate univentionstaff 2015-10-26 17:41:14 CET
Package rebuilt with patch applied.
Comment 7 Daniel Tröder univentionstaff 2015-11-02 15:33:52 CET
OK: containers run shut down from within, are killed only when they are done shutting down
OK: output of init scripts for docker-app-containers and docker.io
OK: docker containers restart cleanly
Comment 8 Stefan Gohmann univentionstaff 2015-11-17 12:11:37 CET
UCS 4.1 has been released:
 https://docs.software-univention.de/release-notes-4.1-0-en.html
 https://docs.software-univention.de/release-notes-4.1-0-de.html

If this error occurs again, please use "Clone This Bug".
Comment 9 Daniel Tröder univentionstaff 2017-05-02 11:02:43 CEST
*** Bug 38259 has been marked as a duplicate of this bug. ***