Bug 54082 - appcenter/query (docker_bridge_network_conflict): ipaddress.AddressValueError: Expected 4 octets in 'None'
appcenter/query (docker_bridge_network_conflict): ipaddress.AddressValueError...
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: App Center
UCS 5.0
Other other
: P5 normal (vote)
: UCS 5.0-1-errata
Assigned To: Nikola Radovanovic
Dirk Wiesenthal
:
: 54201 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2021-11-18 10:47 CET by Maximilian Janßen
Modified: 2022-01-26 17:21 CET (History)
5 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.091
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2021122221000081, 2021121921000177, 2021121621000021, 2021120921000187, 2021120921000071, 2021120821000161, 2021120621000656, 2021120621000012, 2021120521000416, 2021110321000449, 2021110321000511, 2021110321000485, 2021100521000634, 2021100521000554
Bug group (optional): Error handling
Max CVSS v3 score:


Attachments
shows where the "none" is from (140.06 KB, image/jpeg)
2021-12-24 02:40 CET, Telirand
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Maximilian Janßen univentionstaff 2021-11-18 10:47:16 CET
Version: 5.0-0 errata134

Error: 
Internal server error during "appcenter/query".
Request: appcenter/query

Traceback (most recent call last):
  File "%PY3%/univention/management/console/base.py", line 344, in __error_handling
    six.reraise(etype, exc, etraceback)
  File "%PY3%/six.py", line 693, in reraise
    raise value
  File "%PY3%/univention/management/console/base.py", line 247, in execute
    function.__func__(self, request, *args, **kwargs)
  File "%PY3%/univention/management/console/modules/decorators.py", line 321, in _response
    result = _multi_response(self, request)
  File "%PY3%/univention/management/console/modules/decorators.py", line 181, in _response
    return function(self, request)
  File "%PY3%/univention/management/console/modules/decorators.py", line 443, in _response
    return list(function(self, iterator, *nones))
  File "%PY3%/univention/management/console/modules/decorators.py", line 289, in _fake_func
    yield function(self, *args)
  File "%PY3%/univention/management/console/modules/appcenter/__init__.py", line 388, in query
    if not self._test_for_docker_service():
  File "%PY3%/univention/management/console/modules/appcenter/__init__.py", line 407, in _test_for_docker_service
    if docker_bridge_network_conflict():
  File "%PY3%/univention/appcenter/utils.py", line 92, in docker_bridge_network_conflict
    my_net = ipaddress.IPv4Network(u'%s/%s' % (iface['network'], iface['netmask']), False)
  File "/usr/lib/python3.7/ipaddress.py", line 1528, in __init__
    addr = self._ip_int_from_string(args[0])
  File "/usr/lib/python3.7/ipaddress.py", line 1135, in _ip_int_from_string
    raise AddressValueError("Expected 4 octets in %r" % ip_str)
ipaddress.AddressValueError: Expected 4 octets in 'None'
Comment 1 Telirand 2021-12-06 23:24:13 CET
Sorry i duplicated a bug on this:
Bug 54201
Version 5.0 eratta164
Comment 2 Telirand 2021-12-09 10:00:49 CET
I had seen some comments about it being related to multiple ip addresses on the same NIC (4.3) in the forum.

but on an AD takeover, there is ALWAYS multiple  ip addresses on the same nic.
the univention  "aliases" the old servers ip ,so that when the server is removed it can takeover.
Comment 3 Maximilian Janßen univentionstaff 2021-12-22 08:29:13 CET
reported again:
Version: 5.0-0 errata175
Remark: during install of cups
Comment 4 Philipp Hahn univentionstaff 2021-12-23 11:54:36 CET
*** Bug 54201 has been marked as a duplicate of this bug. ***
Comment 5 Philipp Hahn univentionstaff 2021-12-23 11:55:39 CET
If it is related to multiple IP addresses this is how to setup them on the CLI *bypassing* UCR. The new way would be
> ip addr add 1.2.3.4/24 dev "$iface"
and the old way
> ifconfig eth0:0 2.3.4.5 netmask 255.255.255.0 up
which should be equivalent to modern
> ip addr add 2.3.4.5/24 dev "$iface" label "$iface:0"
> ip lint set "$iface" up
to simulate the case, that the interface has multiple addresses.

Via UCR ist setups additional UCRV `interfaces/eth0_0/…:
# ucr search --brief --non-empty ^interfaces/eth0
interfaces/eth0/address: 10.200.17.38
interfaces/eth0/broadcast: 10.200.17.255
interfaces/eth0/ipv6/acceptRA: false
interfaces/eth0/ipv6/default/address: 1:2:3:4::5
interfaces/eth0/ipv6/default/prefix: 64
interfaces/eth0/netmask: 24
interfaces/eth0/network: 10.200.17.0
interfaces/eth0/start: true
interfaces/eth0/type: static
interfaces/eth0_0/address: 1.2.3.4
interfaces/eth0_0/netmask: 24


Looking at /usr/lib/python3/dist-packages/univention/appcenter/utils.py:88 the code is iterating over all interfaces from univention.config_registry.interfaces.Interfaces().ipv4_interfaces and one of them seems to have None as "network" or "netmask".

# sed -ne '88,95p' /usr/lib/python3/dist-packages/univention/appcenter/utils.py 
> def docker_bridge_network_conflict():
>         docker0_net = ipaddress.IPv4Network(u'%s' % (ucr_get('docker/daemon/default/opts/bip', '172.17.42.1/16'),), False)
>         for name, iface in interfaces.Interfaces().ipv4_interfaces:
>                 if 'network' in iface and 'netmask' in iface:
>                         my_net = ipaddress.IPv4Network(u'%s/%s' % (iface['network'], iface['netmask']), False)
>                         if my_net.overlaps(docker0_net):
>                                 return True
>         return False

python3 -c '__import__("pprint").pprint(list(__import__("univention.config_registry.interfaces").config_registry.interfaces.Interfaces().ipv4_interfaces))'
[('eth0',
  {'address': '10.200.17.38',
   'broadcast': '10.200.17.255',
   'ipv6/acceptRA': 'false',
   'ipv6/default/address': '1:2:3:4::5',
   'ipv6/default/prefix': '64',
   'name': 'eth0',
   'netmask': '24',
   'network': '10.200.17.0',
   'start': 'true',
   'type': 'static'}),
 ('eth0_0', {'address': '1.2.3.4', 'name': 'eth0_0', 'netmask': '24'})]

Next step: investigate what AD Takeover does to the network configuration stored in UCRV.
Comment 6 Philipp Hahn univentionstaff 2021-12-23 12:00:21 CET
(In reply to Philipp Hahn from comment #5)
> # sed -ne '88,95p'
> /usr/lib/python3/dist-packages/univention/appcenter/utils.py 
> > def docker_bridge_network_conflict():
> >         docker0_net = ipaddress.IPv4Network(u'%s' % (ucr_get('docker/daemon/default/opts/bip', '172.17.42.1/16'),), False)
> >         for name, iface in interfaces.Interfaces().ipv4_interfaces:
> >                 if 'network' in iface and 'netmask' in iface:
> >                         my_net = ipaddress.IPv4Network(u'%s/%s' % (iface['network'], iface['netmask']), False)
> >                         if my_net.overlaps(docker0_net):
> >                                 return True
> >         return False

Checking for "in" is not enough; maybe change it to
> "iface.get("network") and ifave.get("netmask")
or even better
>         return any(
>           docker0_net.overlaps(ipaddress.IPv4Network(u'%(network)s/%(netmask)s' % iface), False))
>           for _, iface in interfaces.Interfaces().ipv4_interfaces
>           if iface.get("network") and iface.get("netmask")
>         )
for both correctness, simplicity and readability.
Comment 7 Telirand 2021-12-24 02:21:57 CET
THANKS!!!!

found it......


root@ad01:/var/log/univention# python3 -c '__import__("pprint").pprint(list(__import__("univention.config_registry.interfaces").config_registry.interfaces.Interfaces().ipv4_interfaces))'
[('ens192',
  {'address': '192.168.0.15',
   'name': 'ens192',
   'netmask': '24',
   'type': 'static'}),
 ('ens192_1',
  {'address': '192.168.0.16',
   'broadcast': 'None',
   'name': 'ens192_1',
   'netmask': '24',
   'network': 'None'})]
root@ad01:/var/log/univention# 

when you do an AD takeover from clean with a SINGLE network card.
you have to be on the same subnet.
the last procedure in hte AD takeover is turn off server
then univention reconfigures the network by adding an alas to the existing network card in hte same subnet with the OLD AD server IP address.

THAT is the record it added  ens192_1
Comment 8 Telirand 2021-12-24 02:32:43 CET
nano  /etc/network/interfaces


source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#allow-hotplug enp0s31f6
#iface enp0s31f6 inet dhcp

auto ens192
iface ens192 inet static
        address 192.168.0.15
        netmask 255.255.255.0
        gateway 192.168.0.252

auto ens192:1
iface ens192:1 inet static
        address 192.168.0.16
        netmask 255.255.255.0

bob22.jpg.....  guess THAT is where it is from.
Comment 9 Telirand 2021-12-24 02:39:17 CET
nano  /etc/network/interfaces


source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#allow-hotplug enp0s31f6
#iface enp0s31f6 inet dhcp

auto ens192
iface ens192 inet static
        address 192.168.0.15
        netmask 255.255.255.0
        gateway 192.168.0.252

auto ens192:1
iface ens192:1 inet static
        address 192.168.0.16
        netmask 255.255.255.0

bob22.jpg.....  guess THAT is where it is from.
Comment 10 Telirand 2021-12-24 02:40:22 CET
Created attachment 10889 [details]
shows where the "none" is from
Comment 11 Nikola Radovanovic univentionstaff 2021-12-24 08:34:50 CET
(In reply to Telirand from comment #10)
> Created attachment 10889 [details]
> shows where the "none" is from

Thank you both. There is a `create_virtual_IP_alias` method that I have to check.
Comment 12 Telirand 2021-12-24 08:36:54 CET
If you want anything run or checked on a "Still broken system"
just ask......
Comment 13 Nikola Radovanovic univentionstaff 2022-01-12 08:55:47 CET
(In reply to Telirand from comment #12)
> If you want anything run or checked on a "Still broken system"
> just ask......

Hi, I am back from vacation, so yes, I would like to use system where it can be reproduced - if not a problem.
Comment 14 Florian Best univentionstaff 2022-01-12 09:14:59 CET
At least the root of the symptoms is that the original interface "ens192" configuration in UCR doesn't have a "network" and "broadcast" set (why not?).
I.e. ucr get interfaces/ens192/network → none

The code in expects these values always to be set:
management/univention-management-console-module-adtakeover/umc/python/adtakeover/takeover.py

1720 »   def create_virtual_IP_alias(self):
…
1739 »   »   »   »   guess_network = self.ucr["interfaces/%s/network" % self.primary_interface]
…
1744 »   »   »   »   »   "univention-config-registry", "set",
…
1746 »   »   »   »   »   "interfaces/%s/network=%s" % (new_interface_ucr, guess_network),

So it should be very easy to reproduce this concrete error/exception.
The question which remains would be why the UCR interface configuration was incomplete?
Comment 15 Telirand 2022-01-12 09:25:22 CET
Strongly suspect it is related to the AD migration. all the problems were on the aliased address.

but 
    going thru & removing the "None" from that solved 99% of all issues, in Samba, the store/printer etc.



    We found for example
    univention-portal->network settings:

    Primary network interface. (has a two line GUI selection)

    1."interface" as in "ens192"
    2. "None"

    we think maybe there is sometimes "leakage" on the GUI ,in other areas.
    and the "None" is leaking in  via this sort of thing.

    but that said.. for the gui, it should not be accepting strings where only numerics are valid.

    Finally in "univention-portal->network settings:"
    There is NO reference to the aliased address inserted by the AD migration.(when using single network cards)

    even though it is part of the IP network interfaces shown in the UCS UCR variables as

    ens192_1

so even in the the GUI it is inconsistent on the information it is displaying.
Comment 16 Nikola Radovanovic univentionstaff 2022-01-12 09:33:35 CET
(In reply to Florian Best from comment #14)
> At least the root of the symptoms is that the original interface "ens192"
> configuration in UCR doesn't have a "network" and "broadcast" set (why not?).
> I.e. ucr get interfaces/ens192/network → none
> 
> The code in expects these values always to be set:
> management/univention-management-console-module-adtakeover/umc/python/
> adtakeover/takeover.py
> 
> 1720 »   def create_virtual_IP_alias(self):
> …
> 1739 »   »   »   »   guess_network = self.ucr["interfaces/%s/network" %
> self.primary_interface]
> …
> 1744 »   »   »   »   »   "univention-config-registry", "set",
> …
> 1746 »   »   »   »   »   "interfaces/%s/network=%s" % (new_interface_ucr,
> guess_network),
> 
> So it should be very easy to reproduce this concrete error/exception.
> The question which remains would be why the UCR interface configuration was
> incomplete?

I guess that _get_primary_interface in the very same class returns None for some reason. When this happens, maybe we should throw an exception with additional info. I tried to reproduce with my own VM but failed, probably not doing something right. Is it possible to use system where it can be reproduced so I can plant breakpoint(s) and/or put additional logs?
Comment 17 Florian Best univentionstaff 2022-01-12 09:40:08 CET
(In reply to Nikola Radovanovic from comment #16)
> I guess that _get_primary_interface in the very same class returns None for
> some reason.

I doubt this. I think `self.ucr["interfaces/%s/network" % self.primary_interface]` returns None independent of `self.primary_interface`.
Otherwise we would see the `network` as part of comment 7.
Comment 18 Telirand 2022-01-12 09:49:48 CET
I only found it because of your code:
python3 -c '__import__("pprint").pprint(list(__import__("univention.config_registry.interfaces").config_registry.interfaces.Interfaces().ipv4_interfaces))'


maybe it is related to the network driver on vm's or the version of the VMX host
in which case it's not going to to be easy to find.
or it might be the result of using "back/cancel" during the initial install.

Just throw an error as soon as the field is bad & before it is reused, to stop unrelated errors
Comment 19 Nikola Radovanovic univentionstaff 2022-01-12 09:54:45 CET
(In reply to Telirand from comment #18)
> I only found it because of your code:
> python3 -c
> '__import__("pprint").pprint(list(__import__("univention.config_registry.
> interfaces").config_registry.interfaces.Interfaces().ipv4_interfaces))'
> 
> 
> maybe it is related to the network driver on vm's or the version of the VMX
> host
> in which case it's not going to to be easy to find.
> or it might be the result of using "back/cancel" during the initial install.
> 
> Just throw an error as soon as the field is bad & before it is reused, to
> stop unrelated errors

The only notable difference is usage of "new" interface naming nomenclature, which might be caused by some not yet observed configuration. I think there is also a minor bug in IPv6 (not related to this). Not sure how to tackle this. I would personally try with planting breakpoint which is easy, if we have a system that can reliable reproduce
Comment 20 Maximilian Janßen univentionstaff 2022-01-14 11:46:47 CET
reported again: 2022011321000554
Version: 5.0-1 errata182

reported again: 2022011321000563
Version: 5.0-1 errata182
Comment 22 Dirk Wiesenthal univentionstaff 2022-01-26 03:24:56 CET
univention-appcenter.yaml
a3f07aaeda8e | Bug #54082: appcenter query docker bridge network conflict

univention-appcenter (9.0.2-70)
a3f07aaeda8e | Bug #54082: appcenter query docker bridge network conflict

Package: univention-appcenter
Version: 9.0.2-70A~5.0.0.202201260307
Branch: ucs_5.0-0
Scope: errata5.0-1

Code review: OK
YAML: OK
Tests: OK