Univention Bugzilla – Bug 43455
Running systemd inside container needs special docker run options
Last modified: 2017-09-26 10:47:17 CEST
Lots of postings from 2014 say that running systemd in a docker container requires --privileged or --cap-add=SYS_ADMIN options. This is what happens without it: root@ucs42i1:~# docker run -it debian:jessie /sbin/init & root@ucs42i1:~# docker exec -it 2519ec71451f /bin/bash root@2519ec71451f:/# apt-get update; apt-get install -y openssh-server [...] root@2519ec71451f:/# systemctl start sshd Failed to get D-Bus connection: Unknown error -1 root@2519ec71451f:/# systemctl list-units Failed to get D-Bus connection: Unknown error -1 There is a blog article from Redhat describing how to make it work without granting privileges to the containers: https://developers.redhat.com/blog/2016/09/13/running-systemd-in-a-non-privileged-container/ But I couldn't get it to work in a quick test. Also for --cap-add=SYS_ADMIN I had to give more options. This worked for now, but it's not optimal because it's granting privileges: docker run -it --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro --cap-add=SYS_ADMIN debian:jessie /sbin/init
A normal container has these capabilities: ====================================================================== chown, dac_override, fowner, fsetid, kill, setgid, setuid, setpcap, net_bind_service, net_raw, sys_chroot, mknod, audit_write, setfcap ====================================================================== --cap-add=SYS_ADMIN adds just that. On the other hand a --privileged container gets: ====================================================================== chown, dac_override, dac_read_search, fowner, setgid, setuid, sys_ptrace, sys_admin, audit_control, syslog, cap_37 ====================================================================== For our own UCS containers we may be able to adjust systemd configuration inside the container, see https://github.com/maci0/docker-systemd-unpriv/blob/master/Dockerfile ## So, with this we can do now with docker 1.12.1: docker run -d --stop-signal SIGRTMIN+3 --tmpfs /run --tmpfs /tmp -v /sys/fs/cg roup:/sys/fs/cgroup:ro -e 'container=docker' --cap-add=SYS_ADMIN debian:jessie /sbin/init docker exec -it ed352153eaf4 systemctl status docker stop ed352153eaf4
Created attachment 8514 [details] systemd_docker_create_options.patch First sketch for an appcenter patch
Ok, I've taken a different approach than the --cap-add: univention-docker now installs a slightly adjusted seccomp profile and univention-appcenter-docker passes a couple of options to the docker create command, one of which is this adjusted seccomp profile. We now also mount a couple of additional directories into the container, which are required by a standard Debian jessie. Unfortunately I could not test with a UCS 4.2 appbox yet (Bug 42038). We don't specify the --stop-signal, this may be done per image via Dockerfile STOPSIGNAL directive.
Maybe we can adjust some things inside the systemd-based container so we can get rid of some of the additional pre-mounts, I've added notes to Bug 42038. Please note, that according to this article, it's at least debatable if avoiding giving CAP_SYS_ADMIN to a systemd based container makes things more safe: * https://www.freedesktop.org/wiki/Software/systemd/ContainerInterface/ It also explains the purpose of the "container" environment variable.
I'm currently unable to install the owncloud docker image: [...] Digest: sha256:783946bce673ce72ed6735a7f548440cb6f13ac019a5979684a3c0763b12c209 Status: Downloaded newer image for docker.software-univention.de/ucs-appbox-amd64:4.1-3 Initializing app image docker: opening seccomp profile (/etc/docker/systemd.json) failed: open /etc/docker/systemd.json: no such file or directory. See 'docker create --help'. Going to remove ownCloud (9.1.1-20170120) [...] Running 98univention-pkgdb-tools.inst skipped (already executed) Command '['docker', 'create', '--hostname', 'owncl-40093747', '--env-file', u'/var/lib/univention-appcenter/apps/owncloud82/owncloud82.env', '-p', '40000:80/tcp', '-p', '40001:443/tcp', '-v', u'/var/lib/univention-appcenter/apps/owncloud82/data:/var/lib/univention-appcenter/apps/owncloud82/data', '-v', '/dev/hugepages:/dev/hugepages', '-v', '/sys/fs/cgroup:/sys/fs/cgroup:ro', '-v', u'/var/lib/univention-appcenter/apps/owncloud82/conf:/var/lib/univention-appcenter/apps/owncloud82/conf', '-v', '/sys/fs/fuse/connections:/sys/fs/fuse/connections', '-v', u'/var/lib/owncloud:/var/lib/owncloud', '--tmpfs', '/run', '--tmpfs', '/run/lock', '--security-opt', 'seccomp:/etc/docker/systemd.json', '-e', 'container=docker', 'docker.software-univention.de/ucs-appbox-amd64:4.1-3', '/sbin/init']' returned non-zero exit status 125 Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/__init__.py", line 233, in call_with_namespace result = self.main(namespace) File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install.py", line 70, in main return self.do_it(args) File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install_base.py", line 110, in do_it self._do_it(app, args) File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_install.py", line 67, in _do_it ret = super(Install, self)._do_it(app, args) File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install.py", line 87, in _do_it if self._install_app(app, args): File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_install.py", line 61, in _install_app self._start_docker_image(app, hostdn, password, args) File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_base.py", line 199, in _start_docker_image container = docker.create(hostname, set_vars) File "/usr/lib/pymodules/python2.7/univention/appcenter/docker.py", line 358, in create container = create(self.image, command, hostname, ports, volumes, env_file, args) File "/usr/lib/pymodules/python2.7/univention/appcenter/docker.py", line 203, in create return check_output(['docker', 'create'] + _args + [image] + command).strip() File "/usr/lib/python2.7/subprocess.py", line 573, in check_output raise CalledProcessError(retcode, cmd, output=output) CalledProcessError: Command '['docker', 'create', '--hostname', 'owncl-40093747', '--env-file', u'/var/lib/univention-appcenter/apps/owncloud82/owncloud82.env', '-p', '40000:80/tcp', '-p', '40001:443/tcp', '-v', u'/var/lib/univention-appcenter/apps/owncloud82/data:/var/lib/univention-appcenter/apps/owncloud82/data', '-v', '/dev/hugepages:/dev/hugepages', '-v', '/sys/fs/cgroup:/sys/fs/cgroup:ro', '-v', u'/var/lib/univention-appcenter/apps/owncloud82/conf:/var/lib/univention-appcenter/apps/owncloud82/conf', '-v', '/sys/fs/fuse/connections:/sys/fs/fuse/connections', '-v', u'/var/lib/owncloud:/var/lib/owncloud', '--tmpfs', '/run', '--tmpfs', '/run/lock', '--security-opt', 'seccomp:/etc/docker/systemd.json', '-e', 'container=docker', 'docker.software-univention.de/ucs-appbox-amd64:4.1-3', '/sbin/init']' returned non-zero exit status 125
I fixed the path. To check, I've run univention-app install owncloud82 and the container came up and shows the adjusted seccomp profile: docker inspect 87662afc53f9 | jq '.[0].HostConfig.SecurityOpt[]' 2>&1 | sed 's/"seccomp=\(.*\)"/\1/;s/\\//g' | jq '.syscalls | map(select(.name == "name_to_handle_at")) [ { "name": "name_to_handle_at", "action": "SCMP_ACT_ALLOW", "args": [] } ]
# docker with systemd specific options: -> docker run -v /var/tmp/test:/tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro -e container=docker --serity-opt seccomp:/etc/docker/seccomp-systemd.json --tmpfs /run --tmpfs /run/lock debian /sbin/init -> docker exec -it tiny_khorana journalctl Mar 21 15:55:31 5eaa040083f2 systemd-journal[18]: Runtime journal is using 6.2M (max allowed 49.8M, trying to leave 74.7M free of 492.0M available → current limit 49.8M). Mar 21 15:55:31 5eaa040083f2 systemd-journal[18]: Runtime journal is using 6.2M (max allowed 49.8M, trying to leave 74.7M free of 492.0M available → current limit 49.8M). Mar 21 15:55:31 5eaa040083f2 systemd-journal[18]: Journal started Mar 21 15:55:31 5eaa040083f2 systemd[1]: Starting Slices. Mar 21 15:55:31 5eaa040083f2 systemd[1]: Reached target Slices. ... # 4.2 appbox with out systemd options -> docker run docker-test.software-univention.de/ucs-appbox-amd64:4.2-0-ms2 /sbin/init -> docker exec -it cocky_jennings systemctl status Failed to get D-Bus connection: Unknown error -1 # 4.2 appbox with the systemd options -> docker run -v /sys/fs/cgroup:/sys/fs/cgroup:ro --tmpfs /run --tmpfs /run/lock -e container=docker --security-opt seccomp:/etc/docker/seccomp-systemd.json docker-test.software-univention.de/ucs-appbox-amd64:4.2-0-ms2 /sbin/init -> docker exec -it focused_carson systemctl status WARNING: terminal is not fully functional <E2><97><8F> a39139aa0677 State: degraded Jobs: 0 queued Failed: 2 units Since: Tue 2017-03-21 23:25:40 UTC; 1min 50s ago CGroup: /docker/a39139aa06779132d4703efb91b4da312aca7181fb327732d1f4a38b73e8ec30 <E2><94><9C><E2><94><80> 1 /sbin/init <E2><94><9C><E2><94><80>421 systemctl status <E2><94><9C><E2><94><80>425 systemctl status <E2><94><94><E2><94><80>system.slice <E2><94><9C><E2><94><80>univention-runit.service <E2><94><82> <E2><94><94><E2><94><80>215 runsvdir -P /etc/service log: .......................................................................... <E2><94><9C><E2><94><80>nscd.service <E2><94><82> <E2><94><94><E2><94><80>232 /usr/sbin/nscd <E2><94><9C><E2><94><80>cron.service <E2><94><82> <E2><94><94><E2><94><80>177 /usr/sbin/cron -f <E2><94><9C><E2><94><80>inetd.service <E2><94><82> <E2><94><94><E2><94><80>176 /usr/sbin/inetd -i <E2><94><9C><E2><94><80>atd.service <E2><94><82> <E2><94><94><E2><94><80>178 /usr/sbin/atd -f # container for new App has systemd options -v /sys/fs/cgroup:/sys/fs/cgroup:ro --tmpfs /run --tmpfs /run/lock -e container=docker --security-opt seccomp:/etc/docker/seccomp-systemd.json -> docker inspect compas... "Mounts": [... { "Source": "/sys/fs/cgroup", "Destination": "/sys/fs/cgroup", "Mode": "ro", "RW": false, "Propagation": "rprivate" } "Env": [ ... "container=docker", "SecurityOpt": [ "seccomp={\"defaultAction\":\"SCMP_ACT_ERRNO\",\"architectures\":[\"SCMP_ARCH_X86_64\",\"SCMP_ARCH_X86\",\"SCMP_ARCH_X32\"],\"syscalls\":[{\"name\":\"accept\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"accept4\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"access\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"alarm\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"bind\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"brk\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"capget\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"capset\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"chdir\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"chmod\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\"name\":\"chown\",\"action\":\"SCMP_ACT_ALLOW\",\"args\":[]},{\" "Tmpfs": { "/run": "", "/run/lock": "" },
UCS 4.2 has been released: https://docs.software-univention.de/release-notes-4.2-0-en.html https://docs.software-univention.de/release-notes-4.2-0-de.html If this error occurs again, please use "Clone This Bug".