Univention Bugzilla – Bug 50088
iptables rules broken after reboot for docker-compose (e.g. Guacamole)
Last modified: 2019-10-02 15:55:00 CEST
As explained in https://help.univention.com/t/docker-bridge-networks-kaputte-ip-tables/12735 this problem can be easily reproduced with Guacamole App. Before first reboot (after installation of Guacamole): Administrator@ucs-3795:~$ sudo iptables -t nat -L DOCKER -v Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 2 132 RETURN all -- br-86c320f84dbf any anywhere anywhere 0 0 RETURN all -- docker0 any anywhere anywhere 0 0 DNAT tcp -- !br-86c320f84dbf any anywhere anywhere tcp dpt:40001 to:172.18.0.3:8080 Administrator@ucs-3795:~$ sudo docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 72adf4194b55 docker.software-univention.de/guacamole-guacamole:0.9.13-univention13 "/opt/guacamole/bi..." 3 minutes ago Up 3 minutes 0.0.0.0:40001->8080/tcp guacamole_guacamole_1 efa93e6c8933 docker.software-univention.de/guacamole-guacd:0.9.13-univention13 "/usr/local/sbin/g..." 3 minutes ago Up 3 minutes 4822/tcp guacamole_guacd_1 After reboot: Administrator@ucs-3795:~$ sudo iptables -t nat -L DOCKER -v Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 RETURN all -- docker0 any anywhere anywhere 0 0 DNAT tcp -- !docker0 any anywhere anywhere tcp dpt:40001 to::8080 Administrator@ucs-3795:~$ sudo docker ps -a [sudo] Passwort für Administrator: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 72adf4194b55 docker.software-univention.de/guacamole-guacamole:0.9.13-univention13 "/opt/guacamole/bi..." 12 minutes ago Up 49 seconds 0.0.0.0:40001->8080/tcp guacamole_guacamole_1 efa93e6c8933 docker.software-univention.de/guacamole-guacd:0.9.13-univention13 "/usr/local/sbin/g..." 12 minutes ago Up 49 seconds 4822/tcp guacamole_guacd_1 Containers created via docker-compose have their own network, e.g. br-86c320f84dbf in this case. The iptables rules after reboot don't contain any reference to this network anymore. Could /etc/security/packetfilter.d/20_docker.sh be the culprit in some way? In the forum thread mentioned above it a docker-compose.yml shown where its iptable rules are completely broken after reboot (and even without reboot after some time, real trigger is currently unknown).
Created attachment 10170 [details] univention-firewall docker rules patch This patch sets up the Docker rules as Docker version 18.09.7 on Ubuntu 18.04 does. Currently, I just compared the iptables rules. Further tests will be done later (maybe tomorrow). Some additional comments from my side to the patch: 1. I set up the DOCKER-ISOLATION-STAGE-1/2 chains as newer Docker version does. According to https://github.com/docker/libnetwork/commit/1c73b1c99c14d7f048a2318a3caf589865c76fad it should be more efficient. Maybe this has negative side effects, when the UCS docker manipulates iptables during container starting before univention-firewall is restarted. 2. I'm unsure about the MySQL rules. 3. I don't know if 20_docker.sh did further things intentionally different than the original Docker. 4. The appcenter containers often bind their ports to 0.0.0.0. Then a reverse proxy configuration is set up for the web front end. So it is not necessary to have this port reachable via all external interfaces. Then the Docker iptables rules for port forwarding are not necessary, because the docker-proxy process forwards the local connections (see https://windsock.io/the-docker-proxy/). I welcome every comment.
Created attachment 10176 [details] univention-firewall docker rules patch V2 This is a new patch version with the following changes: - Previous patch file was wrong, because the original file included some changes already - Fixed iptables rules when docker port mappings are bind to specific host IP addresses only Furthermore, I did some tests with my docker containers. Especially, the one created with docker-compose.yml shown in https://help.univention.com/t/docker-bridge-networks-kaputte-ip-tables/12735 works know as expected.
After some consideration Johannes and I decided to use the patch with some minor tweaks. What the file does is that it tries to recreate the iptables rules that docker uses, as the iptables rules get flushed everytime univention-firewall gets restarted. With future docker versions the iptables rules may become inconsistent again. I wrote a test so we notice when that happens. Successful build Package: univention-firewall Version: 11.0.1-5A~4.4.0.201909251451 Branch: ucs_4.4-0 Scope: errata4.4-2 0fa60ff797 (HEAD -> 4.4-2, origin/4.4-2) Bug #50088: yaml 9e403e9136 Bug #50088: added test for docker iptables rules 7d9a354fe8 Bug #50088: yaml 4376fb1b81 Bug #50088: changelog 57e491bde0 Bug #50088: Expanded iptables settings to correctly work with docker compose
Thanks for integrating the patch. https://github.com/univention/univention-corporate-server/blob/4.4-2/base/univention-firewall/conffiles/etc/security/packetfilter.d/20_docker.sh looks good. Now, I'm waiting for official release of the updated package. At least I did the update to 4.4-2 today.
(In reply to Daniel Krüger from comment #5) > Thanks for integrating the patch. > https://github.com/univention/univention-corporate-server/blob/4.4-2/base/ > univention-firewall/conffiles/etc/security/packetfilter.d/20_docker.sh looks > good. > > Now, I'm waiting for official release of the updated package. At least I did > the update to 4.4-2 today. Thanks for providing the patch. If not already done, please sign the contributor agreement: https://www.univention.com/about-us/open-source/contributor-agreement/ Thank you very much.
Contribution agreement has arrived and been processed yesterday. No blocker on this side.
OK: rules are persistent OK: no error when docker is not running / some images are not up OK: YAML OK: test
<http://errata.software-univention.de/ucs/4.4/295.html>