Bug 44468 - Create updated UCS docker image
Create updated UCS docker image
Status: RESOLVED FIXED
Product: UCS
Classification: Unclassified
Component: Docker
UCS 4.1
Other Linux
: P5 normal with 4 votes (vote)
: ---
Assigned To: Arvid Requate
Dirk Wiesenthal
https://hub.docker.com/u/univention/
:
Depends on: 43455 45040 45172
Blocks: 46665
  Show dependency treegraph
 
Reported: 2017-04-25 17:00 CEST by Erik Damrose
Modified: 2018-03-14 19:28 CET (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 6: Setup Problem: Issue for the setup process
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.171
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Erik Damrose univentionstaff 2017-04-25 17:00:03 CEST
New UCS docker images should be created, the current version is UCS 4.1-3 e234. As bug 41478 shows, the current image is buggy, and it is rather old.

The new image could be UCS 4.2, or should we build an updated 4.1 image?

+++ This bug was initially created as a clone of Bug #41478 +++
Comment 1 Arvid Requate univentionstaff 2017-04-25 18:14:33 CEST
We should build both new IMHO

* UCS 4.1-4 e410
* UCS-4.2-0

Building 4.1-4 should be just a matter of running the existing univention-docker-dev scripts in the right order. For the UCS 4.2-0 image we should check how we can reduce unneeded packages.
Comment 2 Dirk Wiesenthal univentionstaff 2017-04-25 18:57:30 CEST
Although not e410, 4.1-4 already exists. I think it is not yet through QA.
Comment 3 Arvid Requate univentionstaff 2017-04-25 22:26:52 CEST
Ok, which UCS docker images are we talking about here? If this is about the appbox image, then it's a duplicate of Bug 42038. Or are we talking about the images on hub.docker.com?
Comment 4 Stefan Gohmann univentionstaff 2017-04-26 06:04:42 CEST
The bug is about the images on hub.docker.com.
Comment 5 Arvid Requate univentionstaff 2017-05-15 19:47:28 CEST
I've adjusted the scripts in ucs-4.1/internal/univention-docker-dev and generated 4.2-0 images (generic, master, backup, slave, member) but they are rather large. I think we should shrink them before uploading.


When uploading the description of the images needs to be updated to mention the additional options required to run a UCS 4.2 container (containing systemd):

  --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
  -e 'container=docker' --cap-add=SYS_ADMIN 

Also, the --tmpfs option only exists in more recent docker versions (1.10).
Comment 6 Christian Günther 2017-05-30 17:51:55 CEST
I was able to upgrade my docker container from 4.1 to 4.2. I also noticed the problem of additional necessary container rights and cgroup mapping to be able to handle systemd.

Nevertheless restarting the container involves a long freeze. The univention-docker-container-mode.service uses 4m30secs for startup and blocks the apache2 start. So after a reboot the container is nearly 5min not reachable.

My journalctl -u univention-docker-container-mode.service output shows the following:

May 30 17:10:58 ucs systemd[1]: Starting LSB: Univention Docker Container Mode...
May 30 17:10:58 ucs univention-docker-container-mode[341]: Starting Univention Docker Container Mode.
May 30 17:15:19 ucs univention-docker-container-mode[341]: run-parts: executing /etc/univention/docker/init.d/50univention-mail-postfix
May 30 17:15:19 ucs univention-docker-container-mode[341]: failed.
May 30 17:15:19 ucs systemd[1]: Started LSB: Univention Docker Container Mode.

I do not know what exactly is blocking this long time. But it seems to be obvious that something is crashing there. Is that solved by using a clean 4.2 dockerhub image? When will it be available?
Comment 7 Christian Günther 2017-06-26 23:20:55 CEST
@Arvid Requate: Are you able to provide your modified ucs-4.1/internal/univention-docker-dev package / scripts? Or please describe at least, what you needed to modify.
Comment 8 Christian Günther 2017-06-27 12:17:32 CEST
I was able to hack some additional version checking stuff into the scripts and created a 4.2-1 ucs master image as described here: 

https://forge.univention.org/bugzilla/show_bug.cgi?id=39870

I uploaded the image as cguenther/ucs-master-amd64:4.2-1 to dockerhub. Nevertheless the image seems to be broken. The system does not start up correctly. Even with all capabilities granted by docker, i get still systemd errors for univention-maintenance:

Jun 27 10:08:57 ucs univention-maintenance[116]: Checking network for Univention maintenance...ldap[ds1]...failed.
Jun 27 10:08:57 ucs systemd[1]: univention-maintenance.service: control process exited, code=exited status=1
Jun 27 10:08:57 ucs systemd[1]: Failed to start LSB: Univention Updater.
Jun 27 10:08:57 ucs systemd[1]: Unit univention-maintenance.service entered failed state.

Isn't there someone able to publish prebuild 4.2 images or a guideline how to build them?
Comment 9 Christian Günther 2017-06-27 17:43:07 CEST
The broken univention-maintenance is also reported here:
https://forge.univention.org/bugzilla/show_bug.cgi?id=42038

What is the univention-maintenance for? Is it a problem, when this does not start up on container startup?

Nevertheless, this is not the reason, why the browser based management does not start at system boot. The real reason is that necessary univention packages are not installed after clean image creation.

I runned in the container:
> univention-install univention-docker-container-mode.service 
> #ruint fails during first installation, because of not correctly runned and 
> #configured runit. Removing and reinstalling fixes this problem
> univention-install univention-runit
> univention-remove univention-runit
> univention-install univention-runit
> univention-install univention-management-console-web-server
> apt-get clean

I uploaded this state at dockerhub as cguenther/ucs-master-amd64:4.2-1
It seems that the setup wizard is not present and therefor i have to configure the system with the cli ucs profiles:
http://docs.software-univention.de/installation-4.0.html#appliance:use:auto

Does anyone have an idea how to place the setup_wizard in the container autostart?
Comment 10 Arvid Requate univentionstaff 2017-07-27 16:02:08 CEST
I've created the new images of UCS 4.2-1 with version/erratalevel: 118 and uploaded them to: https://hub.docker.com/u/univention/

I've updated the documentation about the docker run options required for systemd
and explained what the additional capability is required for (see Bug 43455).

The :latest tag has been re-assigned to point to the :4.2-1 tag.


For transparency, I've copied the univention-docker-dev_0.1.0-87_all.deb Debian package into the /root directory of the "stem" based containers (e.g. master), so it's there for inspection:

  dpkg -x univention-docker-dev_*_all.deb .



Please note that the images can also be run interactively in the foreground, which gives a familiar VM-style boot experience which finally presents a login prompt:

docker run -it --hostname=master \
    -e domainname=testdomain.local \
    -e nameserver1="$(ucr get nameserver1)" \
    -e rootpwd=whatyoulike -p 8011:80 \
    -e container=docker \
    -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
    --tmpfs /run --tmpfs /run/lock \
    --cap-add=SYS_ADMIN \
    univention/ucs-master-amd64

Please be aware that a container started this way can only be exited by running the "halt" command. I don't know of any other way to detach that terminal.
Comment 11 Christian Günther 2017-07-28 14:07:38 CEST
Thanks for the progress. I will test it soon to migrate to the official 4.2
Comment 12 Arvid Requate univentionstaff 2017-08-07 17:58:37 CEST
Maybe we should set appcenter/docker=false by default in the images?
Comment 13 Arvid Requate univentionstaff 2017-08-07 19:14:31 CEST
Also univention-portal is currently not installed in the master image, which leads to a traceback with apps that try to register themselves with the portal (because cn=portal is missing). How's the theory around this, I would assume that some dependency mechanism should be in place to pull that package automatically when such an app is installed? (Or the app stops attempting to register if no portal is found?). In absence of this (QA: If you consider that a bug too then please create one), should we simply add the univention-portal into the master (and generic) docker images at build-time?
Comment 14 Dirk Wiesenthal univentionstaff 2017-08-17 17:00:03 CEST
OK, works really nice.

A few things, though.

1) Please set appcenter/docker=false
2) Please rebuild the images because performance improvements have been made
3) Reboot is not really possible. As a memberserver, UMC asks to reboot the system after the join. But it actually shuts down the system. One has to start the container again via "docker start". Not a big deal, but maybe you can come up with a solution?
4) Reboot resets the nameserver1 variable to the one that was initially set during container creation. This is a problem especially for the DC Master which wants to have himself as the nameserver1, not the intial Docker Host
Comment 15 Arvid Requate univentionstaff 2017-08-17 19:11:33 CEST
Ok, will do that. To solve point 3) we can simply add "--restart unless-stopped" to the docker run command in the image description on docker hub. But before I do that we will need to fix 4), otherwise the master's nameserver1 will get reset to the external one directly during first reboot.
I guess I'll propose a fix for Bug 45172 and directly apply that in the new images.
Comment 16 Dirk Wiesenthal univentionstaff 2017-08-17 19:25:55 CEST
While adjusting the README on Docker Hub, please also change the default Domain to testdomain.intranet. You used .lokal or .internet
Comment 17 Arvid Requate univentionstaff 2017-08-21 12:57:11 CEST
> While adjusting the README on Docker Hub, please also change the default Domain to testdomain.intranet. You used .lokal or .internet

I fixed that right away.
Comment 18 Arvid Requate univentionstaff 2017-10-25 15:39:42 CEST
I've built and uploaded all images for UCS 4.2-2 with the adjustments proposed in Comment 14. The images contain a new package version of univention-docker-container-mode, which fixes Bug 45172.
Comment 19 Christian Günther 2017-11-01 12:10:19 CET
Do i have to rebuild my container with your new docker image, or is it sufficient to update via the ucs package updater to benefit from these fixes?
Comment 20 Arvid Requate univentionstaff 2017-11-09 18:34:09 CET
Christian Günther asked on 2017-11-01 12:10:19 CET:

> Do i have to rebuild my container with your new docker image, or is it
> sufficient to update via the ucs package updater to benefit from these fixes?

Sorry for the latency in answering your question. Today we released an updated univention-docker-container-mode Debian package which should be installable via normal package update in the container. After doing that a container restart should not reset the UCR variable nameserver1 any more the the value given during docker run -e nameserver1=... You may check the value before and after running the restrart.


@dwiesenthal: As discussed, I've updated the backup, slave and member images, so they also start system setup during initial UMC login. Documented in

https://git.knut.univention.de/univention/internal/univention-docker-dev/commit/c39d37716435533456bac687996a25686b8c14c1