Bug 43338 - Docker App installation fails in UCS 4.2
Docker App installation fails in UCS 4.2
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Docker
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.2
Assigned To: Dirk Wiesenthal
Felix Botner
: interim-1
Depends on:
Blocks: 43674
  Show dependency treegraph
 
Reported: 2017-01-14 09:38 CET by Stefan Gohmann
Modified: 2017-04-04 18:28 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
appcenter.log (8.13 KB, text/plain)
2017-01-14 09:39 CET, Stefan Gohmann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2017-01-14 09:38:12 CET
The Docker App installation fails in UCS 4.2 for example the test case 80_docker/51_app_install.

root@master221:~# univention-app install --noninteractive --username=Administrator --pwdfile=/var/lib/ucs-test/pwdfile l35s4yeh3n=7.5.6
Going to install l35s4yeh3n (7.5.6)
Creating data directories for l35s4yeh3n...
Registering UCR for l35s4yeh3n
Marking l35s4yeh3n=7.5.6 as installed
File: /etc/univention/service.info/services/univention-appcenter.cfg
Creating /etc/init.d/docker-app-l35s4yeh3n
Adding localhost to LDAP object
Reloading apache2 configuration (via systemctl): apache2.service.
Registering the container host l35s4-06428204 for l35s4yeh3n
Verifying Docker registry manifest for app image docker.software-univention.de/ucs-appbox-amd64:4.1-0
Downloading app image docker.software-univention.de/ucs-appbox-amd64:4.1-0
4.1-0: Pulling from docker.software-univention.de/ucs-appbox-amd64
11dfab8a205f: Already exists
1d13bc13f3d7: Already exists
15697233729e: Already exists
97bcad21930f: Already exists
Digest: sha256:01902de8e5cfb228508990df468a4d746f29e390a55e73ab5e495100b56b16fa
Status: Image is up to date for docker.software-univention.de/ucs-appbox-amd64:4.1-0
Initializing app image
Preconfiguring container b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101
Starting l35s4yeh3n Container b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101 ....
Going to remove l35s4yeh3n (7.5.6)
b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101
b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101
Removing localhost from LDAP object
File: /etc/univention/service.info/services/univention-appcenter.cfg
Reloading apache2 configuration (via systemctl): apache2.service.
Error sending app infos to the App Center server: HTTP Error 404: Not Found
[Errno 2] No such file or directory: '/var/lib/docker/overlay/b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101/merged/etc/machine.secret'
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/__init__.py", line 191, in call_with_namespace
    result = self.main(namespace)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install.py", line 67, in main
    return self.do_it(args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install_base.py", line 109, in do_it
    self._do_it(app, args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_install.py", line 65, in _do_it
    ret = super(Install, self)._do_it(app, args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install.py", line 82, in _do_it
    if self._install_app(app, args):
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_install.py", line 59, in _install_app
    self._start_docker_image(app, hostdn, password, args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_base.py", line 206, in _start_docker_image
    with open(docker.path('/etc/machine.secret'), 'w+b') as f:
IOError: [Errno 2] No such file or directory: '/var/lib/docker/overlay/b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101/merged/etc/machine.secret'
Traceback (most recent call last):
  File "/usr/bin/univention-app", line 90, in <module>
    main()
  File "/usr/bin/univention-app", line 77, in main
    ret = args.func(args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/__init__.py", line 191, in call_with_namespace
    result = self.main(namespace)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install.py", line 67, in main
    return self.do_it(args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install_base.py", line 109, in do_it
    self._do_it(app, args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_install.py", line 65, in _do_it
    ret = super(Install, self)._do_it(app, args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/install.py", line 82, in _do_it
    if self._install_app(app, args):
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_install.py", line 59, in _install_app
    self._start_docker_image(app, hostdn, password, args)
  File "/usr/lib/pymodules/python2.7/univention/appcenter/actions/docker_base.py", line 206, in _start_docker_image
    with open(docker.path('/etc/machine.secret'), 'w+b') as f:
IOError: [Errno 2] No such file or directory: '/var/lib/docker/overlay/b8afb6a571ef5e5ff3c2cd13b2a86048e2325420d74ff21f119e896901a8c101/merged/etc/machine.secret'
Comment 1 Stefan Gohmann univentionstaff 2017-01-14 09:39:23 CET
Created attachment 8353 [details]
appcenter.log
Comment 2 Stefan Gohmann univentionstaff 2017-01-14 09:42:41 CET
Please have a look if it is an App Center issue.
Comment 3 Florian Best univentionstaff 2017-01-14 11:57:50 CET
And please add an empty UCS 4.2 index.json.gz.gpg to the appcenter.
Comment 4 Florian Best univentionstaff 2017-01-23 14:22:06 CET
I added a SKIP tag to all these tests (except one) until this is fixed to reduce the number of tests I have to open every day.

ucs-test (7.0.10-4):
r76014 | Bug #43338: temporarily disable tests
Comment 5 Dirk Wiesenthal univentionstaff 2017-01-31 12:02:22 CET
The path to the root file system of a container changed between the docker versions. Hopefully
  univention-appcenter 6.0.1-7A~4.2.0.201701311158
now uses a stable API wrt the file system.

Re-enabled the tests.

Changelog adjusted.
Comment 6 Florian Best univentionstaff 2017-02-01 10:00:48 CET
REOPEN: All docker tests failed.
Comment 7 Florian Best univentionstaff 2017-02-01 10:15:10 CET
(In reply to Florian Best from comment #6)
> REOPEN: All docker tests failed.
→ This is Bug #43458
Comment 8 Felix Botner univentionstaff 2017-02-21 11:42:30 CET
please fix the apache templates 

new default sites are
 
 /etc/apache2/sites-available/000-default.conf
 /etc/apache2/sites-available/default-ssl.conf
Comment 9 Dirk Wiesenthal univentionstaff 2017-02-22 02:52:00 CET
Done in
  univention-appcenter 6.0.1-30A~4.2.0.201702220251
Comment 10 Dirk Wiesenthal univentionstaff 2017-02-22 12:18:42 CET
Please verify that no config file /etc/apache2/sites-available/000-default is left by the App Center (when upgrading from 4.1-4, not 4.2-0-dev). This should (?) be handled by univention-apache. If it is not, I need to adjust the App Center again.
Comment 11 Felix Botner univentionstaff 2017-02-27 13:06:55 CET
strange, i have UCS 4.1-4 with one docker app -> update to 4.2 and now the apache configs look like

-> more /etc/apache2/sites-enabled/
000-default.conf               univention.conf                univention-saml                
default-ssl.conf               univention-management-console  univention-saml.conf           
root@master:~# more /etc/apache2/sites-enabled/000-default.conf 
# Warning: This file is auto-generated and might be overwritten by
#          univention-config-registry.
#          Please edit the following file(s) instead:
# Warnung: Diese Datei wurde automatisch generiert und kann durch
#          univention-config-registry überschrieben werden.
#          Bitte bearbeiten Sie an Stelle dessen die folgende(n) Datei(en):
# 
#       /etc/univention/templates/files/etc/apache2/sites-available/000-default.d/00start
#       /etc/univention/templates/files/etc/apache2/sites-available/000-default.d/99end
# 

<VirtualHost *:80>
        Include /etc/apache2/ucs-sites.conf.d

</VirtualHost>

Where is my App? Where is the 10univention-appcenter subtemplate?

-> more /etc/apache2/sites-enabled/default-ssl.conf 
# Warning: This file is auto-generated and might be overwritten by
#          univention-config-registry.
#          Please edit the following file(s) instead:
# Warnung: Diese Datei wurde automatisch generiert und kann durch
#          univention-config-registry überschrieben werden.
#          Bitte bearbeiten Sie an Stelle dessen die folgende(n) Datei(en):
# 
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/00start
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/10hsts
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/10univention-appcenter
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/99end
# 

<IfModule mod_ssl.c>

<VirtualHost *:443>
        Include /etc/apache2/ucs-sites.conf.d
        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile /etc/univention/ssl/master.four.test/cert.pem
        SSLCertificateKeyFile /etc/univention/ssl/master.four.test/private.key
        SSLCACertificateFile /etc/univention/ssl/ucsCA/CAcert.pem

        #SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown

        ### To enable special log format for HTTPS-access
        # LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %p" combinedssl
        # CustomLog /var/log/apache2/access.log combinedssl     ## with port number




</VirtualHost>
</IfModule>

OK 10univention-appcenter subtemplate, but where is my App?

-> more /etc/apache2/sites-available/000-default.conf.debian 
# Warning: This file is auto-generated and might be overwritten by
#          univention-config-registry.
#          Please edit the following file(s) instead:
# Warnung: Diese Datei wurde automatisch generiert und kann durch
#          univention-config-registry überschrieben werden.
#          Bitte bearbeiten Sie an Stelle dessen die folgende(n) Datei(en):
# 
#       /etc/univention/templates/files/etc/apache2/sites-available/default.d/00start
#       /etc/univention/templates/files/etc/apache2/sites-available/default.d/10univention-appcenter
#       /etc/univention/templates/files/etc/apache2/sites-available/default.d/99end
# 

NameVirtualHost *:80
<VirtualHost *:80>
        Include /etc/apache2/ucs-sites.conf.d


        ProxyPass /wordpress http://127.0.0.1:40000/wordpress retry=0
        ProxyPassReverse /wordpress http://127.0.0.1:40000/wordpress


</VirtualHost>

Ah here is my App, but it seems the update of the config file is messed up

-> more /etc/apache2/sites-available/default-ssl.conf.debian 
# Warning: This file is auto-generated and might be overwritten by
#          univention-config-registry.
#          Please edit the following file(s) instead:
# Warnung: Diese Datei wurde automatisch generiert und kann durch
#          univention-config-registry überschrieben werden.
#          Bitte bearbeiten Sie an Stelle dessen die folgende(n) Datei(en):
# 
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/00start
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/10hsts
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/10univention-appcenter
#       /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/99end
# 

<IfModule mod_ssl.c>
NameVirtualHost *:443
<VirtualHost *:443>
        Include /etc/apache2/ucs-sites.conf.d
        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile /etc/univention/ssl/master.four.test/cert.pem
        SSLCertificateKeyFile /etc/univention/ssl/master.four.test/private.key
        SSLCACertificateFile /etc/univention/ssl/ucsCA/CAcert.pem

        #SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown

        ### To enable special log format for HTTPS-access
        # LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %p" combinedssl
        # CustomLog /var/log/apache2/access.log combinedssl     ## with port number



        ProxyPass /wordpress https://127.0.0.1:40001/wordpress retry=0
        ProxyPassReverse /wordpress https://127.0.0.1:40001/wordpress



</VirtualHost>
</IfModule>

dito
Comment 12 Felix Botner univentionstaff 2017-02-27 14:04:49 CET
Also, the docker init scripts seems to be broken

wordpress app already installed
-> univention-app install etherpad-lite
...
insserv: script docker-app-etherpad-lite: service App1_Container already provided!
insserv: exiting now!
update-rc.d: error: insserv rejected the script header

...

-> update-rc.d docker-app-etherpad-lite defaults 41 14
insserv: warning: script 'K01univention-system-setup-boot' missing LSB tags and overrides
insserv: script docker-app-etherpad-lite: service App1_Container already provided!
insserv: exiting now!
update-rc.d: error: insserv rejected the script header


-> ls -la /etc/rc2.d/*docker*
lrwxrwxrwx 1 root root 16 Feb 27 12:36 /etc/rc2.d/S03docker -> ../init.d/docker
lrwxrwxrwx 1 root root 30 Feb 27 12:50 /etc/rc2.d/S04docker-app-wordpress -> ../init.d/docker-app-wordpress


seems that the links in the runlevels are missing.
Comment 13 Dirk Wiesenthal univentionstaff 2017-03-01 03:40:23 CET
Apache Config Files: FIXED
Runlevel: Bug #43674
Comment 14 Felix Botner univentionstaff 2017-03-01 15:41:25 CET
update 4.1 -> 4.2

#       /etc/univention/templates/files/etc/apache2/sites-available/000-default.d/00start
#       /etc/univention/templates/files/etc/apache2/sites-available/000-default.d/99end
#       /etc/univention/templates/files/etc/apache2/sites-available/default.d/10univention-appcenter

This looks strange, i think we have to move the config to the new directory 

conffiles/etc/apache2/sites-available/default.d/
-> conffiles/etc/apache2/sites-available/000-default.d/

added debian/univention-*.maintscript script to move old config

see univention-apache for an example
Comment 15 Dirk Wiesenthal univentionstaff 2017-03-03 04:07:03 CET
I have added a maintscript and harmonized the Apache config file.
Comment 16 Felix Botner univentionstaff 2017-03-21 16:43:03 CET
OK
Comment 17 Stefan Gohmann univentionstaff 2017-04-04 18:28:35 CEST
UCS 4.2 has been released:
 https://docs.software-univention.de/release-notes-4.2-0-en.html
 https://docs.software-univention.de/release-notes-4.2-0-de.html

If this error occurs again, please use "Clone This Bug".