Bug 40515 - Docker bridge 172.17.42.1/16 conflicts with customer network 172.17.0.0/16
Docker bridge 172.17.42.1/16 conflicts with customer network 172.17.0.0/16
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Docker
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.1-2-errata
Assigned To: Felix Botner
Arvid Requate
:
: 42703 (view as bug list)
Depends on: 41596 41597
Blocks: 50627
  Show dependency treegraph
 
Reported: 2016-01-27 14:21 CET by Nico Gulden
Modified: 2019-12-11 13:41 CET (History)
5 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.400
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nico Gulden univentionstaff 2016-01-27 14:21:23 CET
See Ticket#2016012621000471

If a customer has the network 172.17.0.0/16 it will be in conflict with the default docker bridge with 172.17.42.1/16. The network traffic won't work anymore. There is no way to change the bridge address, e.g. via UCR.

Two things should happen:
1) Provide a simple possibility to change the address of the docker bridge to avoid a network conflict.

2) Document and/or notify users with a potential risky network that their network should not match the docker bridge network.

Maybe both things should be done.
Comment 1 Michael Grandjean univentionstaff 2016-02-05 13:32:50 CET
Also Ticket#2016010721000158
Comment 2 Felix Botner univentionstaff 2016-06-16 15:30:36 CEST
univention-docker: r70295, 1.0.1-3.12.201606161523
  * make docker daemon opts configurable (Bug #40515)
  * set defaults 
    docker/daemon/default/opts/bip?"172.17.42.1/16"
    docker/daemon/default/opts/storage-driver?"overlay"  
  * disable docker daemon start if docker_bridge_network_conflict

=> this needs Bug #41596

TODO univention-firewall 
 * use docker/daemon/default/opts/bip for iptables rules

TODO univention-appcenter
 * check docker_bridge_network_conflict in 
   appcenter.Instance._test_for_docker_service
Comment 3 Felix Botner univentionstaff 2016-06-16 15:53:51 CEST
univention-firewall: r70300, 8.0.1-2.86.201606161546
staging/univention-firewall.yaml
 * Use docker bip default for iptables settings in 20_docker.sh
Comment 4 Felix Botner univentionstaff 2016-06-20 13:38:50 CEST
Update 4.1-0:

preup/postup:

* added check_docker_network to 4.1-0 preup (disables docker during
  update to 4.1-0 if a conflict has been deteced)
  * appcenter/docker=no
    docker/autostart=no
    appcenter/docker/update41/disabled=yes
    docker/autostart/update41/disabled=yes
  * ...update41/disabled are used to detect whether docker was disabled
    by univention    

* preup adds a dpkg-divert to docker in order to disable the daemon during
  the udpate (autostart/docker won't work until univention-docker is installed,
  but the docker package starts the daemon before univention-docker)

* postup.sh removes the divert, now autostart/docker is evaluated in the
  init script

=> svn: ucs-4.1-0/base/univention-updater/script r70427
=> script updated in apt.knut.univention.de

univention-docker:
remove ... update41/disabled and restore defaults for docker/autostart and appcenter/docker during this update
Comment 5 Felix Botner univentionstaff 2016-07-12 16:41:36 CEST
univention-appcenter: added docker_bridge_network_conflict to _test_for_docker_service in umc/python/appcenter/__init__.py and display warning if conflict has been detected.


Changes:

univention-docker.yaml
univention-firewall.yaml
univention-appcenter.yaml
ucs-4.1-0/base/univention-updater/script/preup.sh (apt.knut.univention.de)
ucs-4.1-0/base/univention-updater/script/postup.sh (apt.knut.univention.de)

QA:

please verify the changes and that the update from 4.0 to 4.1 disables docker (apt.knut.univention.de). 

Please reopen, i have to copy the update/script changes to the mirror
Comment 6 Stefan Gohmann univentionstaff 2016-07-14 20:59:32 CEST
Code review: OK
 - univention-docker: OK (r70295 + r70417 + r70424)
 - univention-firewall: OK (r70300)
 - univention-appcenter: OK (r70290 + r70948)

YAML: OK
- univention-firewall.yaml: OK
- univention-appcenter.yaml: OK
- univention-docker.yaml: OK


TODOs
-----

Basic tests:
 - univention-docker:
 - univention-firewall:
 - univention-appcenter:

Normal upgrade:

Upgrade with 172.17.0.0/16 network:

Jenkins tests:
Comment 7 Stefan Gohmann univentionstaff 2016-07-15 08:57:22 CEST
Basic tests:
 - univention-docker: OK
 - univention-firewall: OK
 - univention-appcenter: OK

Normal upgrade: OK

Upgrade with 172.17.0.0/16 network: Failed. After changing the docker network and rebooting the system, I'm unable to start new containers. the App Center error message has been removed. From the appcenter.log file:
--------------------------------------------------------------------------------
  4652 actions.install                  16-07-15 02:46:10 [    INFO]: Creating data directories for openproject...
  4652 actions.install                  16-07-15 02:46:10 [    INFO]: Registering the container host openproject-1468565170122216 for openproject
  4652 actions.install.progress         16-07-15 02:46:10 [   DEBUG]: 20
  4652 actions.install                  16-07-15 02:46:10 [    INFO]: Verifying Docker registry manifest for app image docker.software-univention.de/ucs-appbox-amd64:4.1-0
  4652 actions.install                  16-07-15 02:46:11 [    INFO]: Downloading app image docker.software-univention.de/ucs-appbox-amd64:4.1-0
  4652 actions.install                  16-07-15 02:46:46 [    INFO]: Initializing app image
  4652 actions.configure                16-07-15 02:46:46 [   DEBUG]: Finding all configuration options for openproject
  4652 actions.install                  16-07-15 02:46:46 [    INFO]: Preconfiguring container 779159de8babb0764d7f7efeea37f2c557f66bc8c97072563c4fefefd9624a49
  4652 actions.start                    16-07-15 02:46:46 [   DEBUG]: Calling start
  4652 actions.start.progress           16-07-15 02:46:46 [   DEBUG]: 0
  4652 actions.start                    16-07-15 02:46:46 [   DEBUG]: Calling /etc/init.d/docker-app-openproject start
  4886 apps                             16-07-15 02:46:47 [   DEBUG]: Loaded 176 apps from cache
  4886 actions.get                      16-07-15 02:46:47 [   DEBUG]: Calling get
  4886 actions.get.progress             16-07-15 02:46:47 [   DEBUG]: 0
  4886 actions.get                      16-07-15 02:46:47 [    INFO]: appcenter/apps/openproject/container
  4886 actions.get.progress             16-07-15 02:46:47 [   DEBUG]: 100
  4652 actions.start                    16-07-15 02:46:48 [ WARNING]: Error response from daemon: Cannot restart container 779159de8babb0764d7f7efeea37f2c557f66bc8c97072563c4fefefd9624a49: [8]
 System error: invalid argument
  4652 actions.start                    16-07-15 02:46:48 [ WARNING]: time="2016-07-15T02:46:48-04:00" level=fatal msg="Error: failed to restart one or more containers" 
  4652 actions.start                    16-07-15 02:46:48 [    INFO]: Starting openproject Container 779159de8babb0764d7f7efeea37f2c557f66bc8c97072563c4fefefd9624a49 ...
  4652 actions.start                    16-07-15 02:46:48 [   DEBUG]: /etc/init.d/docker-app-openproject returned with 1
  4652 actions.start.progress           16-07-15 02:46:48 [   DEBUG]: 100
  4652 actions.install                  16-07-15 02:46:48 [ WARNING]: Unable to start the container!
  4652 actions.install                  16-07-15 02:46:48 [ WARNING]: Aborting...
  4652 actions.remove                   16-07-15 02:46:48 [   DEBUG]: Calling remove
--------------------------------------------------------------------------------

If you want to use the test system: 10.201.172.1

Upgrade with another network: OK

Jenkins tests: OK
Comment 8 Felix Botner univentionstaff 2016-07-15 11:56:04 CEST
see, https://github.com/docker/docker/issues/14732

 "@timothysparg The issue is here that you can't use 0 as the last octet as it 
  will try to set that as the IP address of the bridge.
  This should be (probably) --bip 192.168.0.1/24"

update docker/daemon/default/opts/bip
-> ucr set docker/daemon/default/opts/bip='172.42.0.1/16' 

seem to work now

-> docker run  hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the
...

Maybe we should check this in the template (disable docker if last octet of bip is 0)?
Comment 9 Stefan Gohmann univentionstaff 2016-07-15 12:19:51 CEST
(In reply to Felix Botner from comment #8)
> see, https://github.com/docker/docker/issues/14732
> 
>  "@timothysparg The issue is here that you can't use 0 as the last octet as
> it 
>   will try to set that as the IP address of the bridge.
>   This should be (probably) --bip 192.168.0.1/24"
> 
> update docker/daemon/default/opts/bip
> -> ucr set docker/daemon/default/opts/bip='172.42.0.1/16' 
> 
> seem to work now

OK, good to know.

> -> docker run  hello-world
> 
> Hello from Docker!
> This message shows that your installation appears to be working correctly.
> 
> To generate this message, Docker took the following steps:
>  1. The Docker client contacted the
> ...
> 
> Maybe we should check this in the template (disable docker if last octet of
> bip is 0)?

Maybe later.

For me the bug is verified. As requested, I reopen the bug.
Comment 10 Felix Botner univentionstaff 2016-07-18 11:09:05 CEST
copied preup and postup to mirror/ftp/4.1/maintained/4.1-0/all
signed both scripts
update from 4.0 to 4.1 with updates.software-univention.de works
Comment 11 Arvid Requate univentionstaff 2016-07-18 13:04:39 CEST
Ok, new preup.sh & postup.sh active online.
Comment 13 Jürn Brodersen univentionstaff 2017-06-27 10:34:46 CEST
*** Bug 42703 has been marked as a duplicate of this bug. ***