Univention Bugzilla – Bug 53273
line break in docker-compose ps subprocess command breaks app installation
Last modified: 2021-09-17 16:31:50 CEST
During the installation of compose apps, at some point the appcenter runs docker-compose ps ... to get the names of the containers. That command automatically adds line breaks if the output becomes to long for the current get_terminal_size() width. The appcenter now sets a wrong container name (rocketchat_rocketchat_ instead of rocketchat_rocketchat_1) and the installation fails. 1261 docker 21-05-17 15:30:48 [ WARNING]: Inspect for main service container rocketchat_rocketchat_ failed: Command '['docker', 'inspect', 'rocketchat_rocketchat_']' returned non-zero exit status 1. 1261 docker 21-05-17 15:30:54 [ WARNING]: Inspect for main service container rocketchat_rocketchat_ failed: Command '['docker', 'inspect', 'rocketchat_rocketchat_']' returned non-zero exit status 1. 1261 docker 21-05-17 15:30:59 [ WARNING]: Inspect for main service container rocketchat_rocketchat_ failed: Command '['docker', 'inspect', 'rocketchat_rocketchat_']' returned non-zero exit status 1. 1261 packages 21-05-17 15:31:05 [ DEBUG]: Releasing LOCK # that is what happens in appcenter/docker.py _get_main_service_container_id -> COLUMNS=80 docker-compose -p rocketchat ps Name Command State Ports -------------------------------------------------------------------------------- rocketchat_mongo-init- docker-entrypoint.sh Exit 0 replica_1 bash ... rocketchat_mongo_1 docker-entrypoint.sh Up 27017/tcp mongo ... rocketchat_rocketchat_ docker-entrypoint.sh Up 0.0.0.0:40003->3000/t 1 bash ... cp vs -> docker-compose -p rocketchat ps Name Command State Ports --------------------------------------------------------------------------------------------------- rocketchat_mongo-init-replica_1 docker-entrypoint.sh bash ... Exit 0 rocketchat_mongo_1 docker-entrypoint.sh mongo ... Up 27017/tcp rocketchat_rocketchat_1 docker-entrypoint.sh bash ... Up 0.0.0.0:40003->3000/tcp simple hack could be -ps = check_output(['docker-compose', '-p', self.app.id, 'ps'], cwd=self.app.get_compose_dir()) +ps = check_output(['docker-compose', '-p', self.app.id, 'ps'], cwd=self.app.get_compose_dir(), env={'COLUMNS': '200'}) but better would be to use the docker python api
That is possibly an old bug in docker-compose: https://github.com/docker/compose/issues/1513, which has been fixed with this: https://github.com/docker/compose/pull/1693/files As I understand it: In a pipe 0 is returned as terminal width, where it used to be 80 before. I would suggest, that we either update docker-compose or better use COLUMNS=0 instead, because docker-compose uses the python-texttable package (https://github.com/foutaise/texttable/) and for that package zero means unlimited table width. Note, that https://github.com/docker/compose/blob/master/compose/cli/formatter.py#L27 is where it gets called.
FYI: I created 3 gitlab comments.
# The "docker-compose API" * A short evaluation let me to the conclusion, that interfacing `docker-compose` seems unreasonable, because it does not have a public API on which one could rely on. * Importing the python classes would be possible, but it would be at least as unstable as its command line interface. * Docker-compose is based upon the API of Docker and we could do the same. It is debatable, but the additional effort is not worth it in my opinion. # Regarding the solution with a command line interface * I rethought my suggestion of setting COLUMNS to zero and I would not further suggest it, because some applications implement different default values for that case. * My fix now makes use of the fact, that within a pipe `COLUMNS` is undefined. I think this is a clean solution, because it is very common for programs to detect if they are running within a pipe and adjust their output accodingly. This would make the fix generic enough to be used in similar cases. # QA Thanks Florian. Your feedback was integrated into the latest change. # Fix Fixed in 9.0.2-47A~5.0.0.202105261442
REOPEN: revert that YAML thing: 480d1892f40214e300b9152f2a1c6234f6a30df9 +++ doc/errata/staging/univention-appcenter.yaml -version: [0] +version: [46] 5e6aae32f4dc1b19c73e5331ff6264a96fe32add +++ doc/errata/staging/univention-appcenter.yaml -version: [46] +version: [47]
QA, please also check if the rocketchat app can be installed
QA: Code review: OK Rocketchat can be installed: OK TODO: Please complete the YAML
Not really sure what is going on here. But the installation of rocketchat failed. 8002 utils 21-05-31 11:53:48 [ INFO]: Creating network "rocketchat_appcenter_net" with the default driver 8002 utils 21-05-31 11:53:49 [ INFO]: Creating rocketchat_mongo_1 ... 8002 utils 21-05-31 11:53:50 [ INFO]: ^[[1A^[[2K^MCreating rocketchat_mongo_1 ... ^[[32mdone^[[0m^M^[[1BCreating rocketchat_rocketchat_1 ... 8002 utils 21-05-31 11:53:50 [ INFO]: Creating rocketchat_mongo-init-replica_1 ... 8002 utils 21-05-31 11:53:53 [ INFO]: ^[[1A^[[2K^MCreating rocketchat_mongo-init-replica_1 ... ^[[32mdone^[[0m^M^[[1B^[[2A^[[2K^MCreating rocketchat_rocketchat_1 ... ^[[32mdone^[[0m^M^[[2B 8002 docker 21-05-31 11:53:56 [ WARNING]: Inspect for main service container rocketchat_rocketchat_ failed: Command '['docker', 'inspect', 'rocketchat_rocketchat_']' returned non-zero exit status 1. 8002 docker 21-05-31 11:54:02 [ WARNING]: Inspect for main service container rocketchat_rocketchat_ failed: Command '['docker', 'inspect', 'rocketchat_rocketchat_']' returned non-zero exit status 1. 8002 docker 21-05-31 11:54:07 [ WARNING]: Inspect for main service container rocketchat_rocketchat_ failed: Command '['docker', 'inspect', 'rocketchat_rocketchat_']' returned non-zero exit status 1. 8002 packages 21-05-31 11:54:12 [ DEBUG]: Releasing LOCK 8002 actions.install 21-05-31 11:54:12 [CRITICAL]: could not find container for service rocketchat! docker-ps: CONTAINER ID also there seems to be a python2 vs python3 issue: UCS 5 after failed rocketchat installation -> cd /var/lib/univention-appcenter/apps/rocketchat/compose -> docker-compose -p rocketchat up -d --no-build --no-recreate -> cat /opt/a from univention.appcenter.docker import MultiDocker from univention.appcenter.app_cache import AppCenterCache, Apps a = AppCenterCache().get_all_apps_with_id('rocketchat') a = AppCenterCache().find_candidate(a[0]) m = MultiDocker(a) container = m._get_main_service_container_id() print(container) # added print(e) after first "except Exception as e:" in _get_main_service_container_id() -> python3 /opt/a 4.4/rocketchat=3.7.1 Error: No such object: rocketchat_rocketchat_ Error: No such object: rocketchat_rocketchat_ Error: No such object: rocketchat_rocketchat_ None -> python2 /opt/a __init__() got an unexpected keyword argument 'input' __init__() got an unexpected keyword argument 'input' __init__() got an unexpected keyword argument 'input' 0ec1a9b220d7d1c6a14138187e9defc737e150337a1f63531f25a538858d35a1
The fix now uses docker-compose .. ps -q. This prints each container ID on a separate line, which causes no line breaks. docker inspect is then called to find the correct image. ace5bceec3 Use docker-compose ps -q 1dc991f2bb yaml univention-appcenter 9.0.2-48A~5.0.0.202106041037
Rocketchat installable: OK Rocketchat installable even if every error is raised and not the default-guess is used: OK Container id is used instead of container name: OK yaml: OK No backport to 4.4 needed: OK Verified
<https://errata.software-univention.de/#/?erratum=5.0x9>
git:ace5bceec3f7a80dfc2254af03dee589344c3fba caused Bug #53803.