Univention Bugzilla – Bug 43612
Add support for docker multi container setup per app in the App Center
Last modified: 2018-12-05 14:39:07 CET
There are several app providers that prefer a docker multi container setup: * Kolab Systems * FileWave * OnlyOffice * M-Way Solutions The following app providers require a multi container setup. A single setup is not supported for production use: * Mattermost
(In reply to Nico Gulden from comment #0) [...] > * OnlyOffice OnlyOffice also requires a multi container setup, as I received a confirmation from the vendor. Therefore multi container setup is a requirement for: * Mattermost * OnlyOffice
* metasfresh ERP Software needs a multi container setup as well
* Zammad would also need multi container support for production use. They have a docker image for testing purposes. https://docs.zammad.org/en/latest/contributing-install-docker.html
Mattermost can be removed from this list. The app was realised in a single container.
Added a first implementation in univention-appcenter 7.0.2-32A~4.3.0.201811011353 Uses docker-compose internally. We need a self-containing docker-compose.yml. Currently, this works with 4.3/zammad=2.6.0-31. See also univention-app internal-transfer-images (on docker)
* docker image/compose-yaml handling is the provider portal is very confusing * port handling also (where should ports be configured, in the yaml compose in the port settings, in the webinterface settings, i could not get it to work.) * no (package) dependency to docker-compose in univention-appcenter * no error handling in MultiDocker * i want to see every process call in the logfile * i want the return value/stderr/stdout of every process call in the logfile (not in the terminal) * raise DockerImagePullFailed if pull fails * raise Docker... if create fails * not sure for up/stop * remove fails univention-app remove ethercalc Going to remove EtherCalc (1.0) Password for Administrator: Configuring ethercalc=1.0 Executing interface configure for ethercalc No interface defined (['docker-compose', '-p', u'ethercalc', 'up', '-d'], u'/var/lib/univention-appcenter/apps/ethercalc/compose') Stopping the container for ethercalc=1.0 failed Could not backup container! Aborting... ethercalc=1.0: Not running, cannot check further "Could not backup container!", what is it trying to do, do we still backup containers? > See also > univention-app internal-transfer-images (on docker) where is that coming from (source package), i can only find /usr/share/pyshared/univention/appcenter/actions/docker_images.py on the docker server
docker-compose and python-ruamel.yaml are still in unmaintained. But the other points have been addressed. Port forwarding and even volumes are now transparent by further changing the compose.yml on the host. New attribute added for multi container apps: DockerMainService can be used (example: zammad-railsserver) instead of DockerImage.
> > See also > > univention-app internal-transfer-images (on docker) > > where is that coming from (source package), i can only find > /usr/share/pyshared/univention/appcenter/actions/docker_images.py on the > docker server https://git.knut.univention.de/univention/components/univention-appcenter-internal/tree/master
I hope I got this right. ruamel.yaml and docker-compose should be maintained. Stolen from Bug#48086
* as discussed, logging needs to be improved * installation/tests fail apt-get install univention-appcenter-docker Paketlisten werden gelesen... Fertig Abhängigkeitsbaum wird aufgebaut. Statusinformationen werden eingelesen.... Fertig Einige Pakete konnten nicht installiert werden. Das kann bedeuten, dass Sie eine unmögliche Situation angefordert haben oder, wenn Sie die Unstable-Distribution verwenden, dass einige erforderliche Pakete noch nicht erstellt wurden oder Incoming noch nicht verlassen haben. Die folgenden Informationen helfen Ihnen vielleicht, die Situation zu lösen: Die folgenden Pakete haben unerfüllte Abhängigkeiten: univention-appcenter-docker : Hängt ab von: docker-compose ist aber nicht installierbar Hängt ab von: python-ruamel.yaml ist aber nicht installierbar E: Probleme können nicht korrigiert werden, Sie haben zurückgehaltene defekte Pakete. root@master:~# apt-get install docker-compose Paketlisten werden gelesen... Fertig Abhängigkeitsbaum wird aufgebaut. Statusinformationen werden eingelesen.... Fertig Paket docker-compose ist nicht verfügbar, wird aber von einem anderen Paket referenziert. Das kann heißen, dass das Paket fehlt, dass es abgelöst wurde oder nur aus einer anderen Quelle verfügbar ist. E: Für Paket »docker-compose« existiert kein Installationskandidat.
(In reply to Felix Botner from comment #10) > * installation/tests fail > > apt-get install univention-appcenter-docker ... > E: Für Paket »docker-compose« existiert kein Installationskandidat. The packages were copied to the errata scope correctly, but the errata scope is announced nightly by "announce_ucs_scope", which honors the "maintained" mechanism: Back when UCS-4.3-2 was releases, "docker-compose" was not a dependency of the "trigger" packages and was thus never added to the list of maintained packages, even now after adding it to "univention-docker". The list of maintained packages must be re-generated explicitly by using the following Jenkins job: <http://jenkins.knut.univention.de:8080/job/Mitarbeiter/job/phahn/job/CalculateMaintained/86/console> I did that and now the package is maintained: # ssh omar grep --color docker /mnt/build-storage/buildsystem/cd-contents/ucs_4.3-2_amd64.maintained # docker-compose After that the errata scope must be re-announced using this Jenkins job: <http://jenkins.knut.univention.de:8080/view/Publish/job/UCS-4.3/job/PublishErrataTest/273/console> I also just did that.
python-cached-property python-websocket python-backports.ssl-match-hostname python-docker python-dockerpty python-docopt python-functools32 python-jsonschema python-texttable We also need these packages in maintained (dependencies of docker-composer).
always (re)generate the internal compose file: /var/lib/univention-appcenter/apps/APP_ID/compose/docker-compose.yml
cd /var/univention/buildsystem2/apt/ucs_4.3-0-errata4.3-2 /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/texttable_0.8.4-2.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/python-jsonschema_2.5.1-6.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/python-functools32_3.2.3.2-3.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.2-0/source/docopt_0.6.2-1.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/dockerpty_0.4.1-1.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/python-docker_1.9.0-1.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/backports.ssl-match-hostname_3.5.0.1-1.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/python-websockets_3.2-2.dsc /home/phahn/bin/repo-copy-dsc -vvvv ../ucs_4.3-0/source/cached-property_1.3.0-2.dsc repo-apt-ftparchive --release "${PWD##*/}" http://jenkins.knut.univention.de:8080/job/Mitarbeiter/job/phahn/job/CalculateMaintained/ http://jenkins.knut.univention.de:8080/view/Publish/job/UCS-4.3/job/PublishErrataTest/
* always create /var/lib/univention-appcenter/apps/ethercalc/compose/docker-compose.yml before running a docker-compose command * remove the /etc/init.d/docker-app-APP_ID stuff
The appcenter tests still fail, please make sure the univention-appcenter-docker installation works: 2018-11-16 01:36:31.037232] ============================= test session starts ============================== [2018-11-16 01:36:31.037357] platform linux2 -- Python 2.7.13, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 [2018-11-16 01:36:31.037396] rootdir: /usr/share/ucs-test/20_appcenter, inifile: [2018-11-16 01:36:31.209423] collected 0 items / 1 errors [2018-11-16 01:36:31.310778] [2018-11-16 01:36:31.310906] ==================================== ERRORS ==================================== [2018-11-16 01:36:31.310967] _______________________ ERROR collecting 100_settings.py _______________________ [2018-11-16 01:36:31.311025] ImportError while importing test module '/usr/share/ucs-test/20_appcenter/100_settings.py'. [2018-11-16 01:36:31.311063] Hint: make sure your test modules/packages have valid Python names. [2018-11-16 01:36:31.311090] Traceback: [2018-11-16 01:36:31.311119] 100_settings.py:23: in <module> [2018-11-16 01:36:31.311150] from univention.appcenter.docker import Docker [2018-11-16 01:36:31.311179] E ImportError: No module named docker [2018-11-16 01:36:31.311234] !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!! [2018-11-16 01:36:31.311261] =========================== 1 error in 0.27 seconds ============================
these packages are still missing in maintained python-ruamel.ordereddict python-typing python-ruamel.yaml
Can not remove a (running) app -> univention-app remove ethercalc Going to remove EtherCalc (1.0) Password for Administrator: No hostdn for ethercalc found. Nothing to remove Configuring ethercalc=1.0 Executing interface configure for ethercalc No interface defined ethercalc is not supported Starting the container for ethercalc=1.0 failed Could not backup container! Aborting... Executing interface update_available for ethercalc No interface defined i think the problem is class Start(Service): '''Starts an application previously installed.''' help = 'Start an app' def main(self, args): if args.app.uses_docker_compose(): docker = MultiDocker(args.app) if not docker.is_running(): return docker.up() return self.call_init(args.app, 'start') if the app is compose and running, i guess call_init is called, should be if compose ... else call_init Please check all other services to
(In reply to Felix Botner from comment #18) > Can not remove a (running) app > > -> univention-app remove ethercalc > Going to remove EtherCalc (1.0) > Password for Administrator: > No hostdn for ethercalc found. Nothing to remove > Configuring ethercalc=1.0 > Executing interface configure for ethercalc > No interface defined > ethercalc is not supported > Starting the container for ethercalc=1.0 failed > Could not backup container! > Aborting... > Executing interface update_available for ethercalc > No interface defined > > i think the problem is > > class Start(Service): > > '''Starts an application previously installed.''' > help = 'Start an app' > > def main(self, args): > if args.app.uses_docker_compose(): > docker = MultiDocker(args.app) > if not docker.is_running(): > return docker.up() > return self.call_init(args.app, 'start') > > > if the app is compose and running, i guess call_init is called, should be if > compose ... else call_init > > Please check all other services to I would also prefer to completely remove the backup stuff.
Another thing, univention-app ethercalc stop removes the containers, is this the new default -> univention-app stop ethercalc Stopping ethercalc_ethercalc_1 ... Stopping ethercalc_redis_1 ... Removing ethercalc_ethercalc_1 ... one Removing ethercalc_redis_1 ... Removing network ethercalc_defaultdone is this the new default (to always re-create the containers upon stop/start)? currently the situation is as follows action start does compose up, which in more like create for docker actiob stop does compose down, which removes everything, a bit like docker remove i think compose start/stop would be more appropriate
There are no default env variables like LDAP_SERVER_NAME etc. Is this correct?
Fixed the start/stop commands, added all the common environment variables (to the main service) univention-appcenter 7.0.2-48A~4.3.0.201811301138
as discussed, upgrade does not work
Fixed by putting the removal of the old container before the installation of the new App. This was not necessary with single container Apps. univention-appcenter 7.0.2-49A~4.3.0.201812032053
OK - docker compose install/upgrade/remove OK - yaml added 80_docker/85_docker_compose
<http://errata.software-univention.de/ucs/4.3/345.html> <http://errata.software-univention.de/ucs/4.3/364.html> <http://errata.software-univention.de/ucs/4.3/365.html> <http://errata.software-univention.de/ucs/4.3/366.html> <http://errata.software-univention.de/ucs/4.3/367.html> <http://errata.software-univention.de/ucs/4.3/368.html> <http://errata.software-univention.de/ucs/4.3/369.html> <http://errata.software-univention.de/ucs/4.3/370.html> <http://errata.software-univention.de/ucs/4.3/371.html> <http://errata.software-univention.de/ucs/4.3/372.html> <http://errata.software-univention.de/ucs/4.3/373.html> <http://errata.software-univention.de/ucs/4.3/374.html> <http://errata.software-univention.de/ucs/4.3/375.html> <http://errata.software-univention.de/ucs/4.3/376.html>