Bug 46003 - 60_umc/05_get_client_ip_address test case fails
60_umc/05_get_client_ip_address test case fails
Status: CLOSED WORKSFORME
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.3
Other Linux
: P5 normal (vote)
: UCS 4.3
Assigned To: Florian Best
Stefan Gohmann
: interim-2
: 45937 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-01-10 15:07 CET by Florian Best
Modified: 2018-03-14 14:38 CET (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2018-01-10 15:07:37 CET
[2018-01-08 23:11:27.636070] *** Entering with-statement of NetworkRedirector()
[2018-01-08 23:11:27.636182] *** Check with different remote addresses
[2018-01-08 23:11:27.636262] *** Adding network loop (1.2.3.4 <--> 4.3.2.1)
[2018-01-08 23:11:27.636401] *** ['/sbin/iptables', '-t', 'mangle', '-A', 'OUTPUT', '-d', '1.2.3.4', '-j', 'TOS', '--set-tos', '0x04']
[2018-01-08 23:11:27.651774] *** ['/sbin/iptables', '-t', 'nat', '-A', 'OUTPUT', '-d', '1.2.3.4', '-j', 'DNAT', '--to-destination', '10.210.16.8']
[2018-01-08 23:11:27.653585] *** ['/sbin/iptables', '-t', 'nat', '-A', 'POSTROUTING', '-m', 'tos', '--tos', '0x04', '-j', 'SNAT', '--to-source', '4.3.2.1']
[2018-01-08 23:11:27.655457] *** ['/sbin/iptables', '-t', 'mangle', '-A', 'OUTPUT', '-d', '4.3.2.1', '-j', 'TOS', '--set-tos', '0x08']
[2018-01-08 23:11:27.665289] *** ['/sbin/iptables', '-t', 'nat', '-A', 'OUTPUT', '-d', '4.3.2.1', '-j', 'DNAT', '--to-destination', '10.210.16.8']
[2018-01-08 23:11:27.666971] *** ['/sbin/iptables', '-t', 'nat', '-A', 'POSTROUTING', '-m', 'tos', '--tos', '0x08', '-j', 'SNAT', '--to-source', '1.2.3.4']
[2018-01-08 23:11:27.669765] 
[2018-01-08 23:11:27.669785] *** UMC request: "POST auth" (None)
[2018-01-08 23:11:27.669796] UMC request payload: 
[2018-01-08 23:11:27.669806] {'options': {'password': 'univention', 'username': 'Administrator'}}
[2018-01-08 23:11:27.670101] UMC request failed: ('Could not send request.', error(111, 'Connection refused'))
[2018-01-08 23:11:27.670506] 
[2018-01-08 23:11:27.670533] *** Leaving with-statement of NetworkRedirector()
[2018-01-08 23:11:27.670550] *** NetworkRedirector.revert_network_settings()
[2018-01-08 23:11:27.670562] *** Removing network loop (1.2.3.4 <--> 4.3.2.1)
[2018-01-08 23:11:27.670573] *** ['/sbin/iptables', '-t', 'mangle', '-D', 'OUTPUT', '-d', '1.2.3.4', '-j', 'TOS', '--set-tos', '0x04']
[2018-01-08 23:11:27.672428] *** ['/sbin/iptables', '-t', 'nat', '-D', 'OUTPUT', '-d', '1.2.3.4', '-j', 'DNAT', '--to-destination', '10.210.16.8']
[2018-01-08 23:11:27.682444] *** ['/sbin/iptables', '-t', 'nat', '-D', 'POSTROUTING', '-m', 'tos', '--tos', '0x04', '-j', 'SNAT', '--to-source', '4.3.2.1']
[2018-01-08 23:11:27.684314] *** ['/sbin/iptables', '-t', 'mangle', '-D', 'OUTPUT', '-d', '4.3.2.1', '-j', 'TOS', '--set-tos', '0x08']
[2018-01-08 23:11:27.686198] *** ['/sbin/iptables', '-t', 'nat', '-D', 'OUTPUT', '-d', '4.3.2.1', '-j', 'DNAT', '--to-destination', '10.210.16.8']
[2018-01-08 23:11:27.688034] *** ['/sbin/iptables', '-t', 'nat', '-D', 'POSTROUTING', '-m', 'tos', '--tos', '0x08', '-j', 'SNAT', '--to-source', '1.2.3.4']
(2018-01-08 23:11:27.689623) Traceback (most recent call last):
(2018-01-08 23:11:27.689649)   File "05_get_client_ip_address", line 54, in <module>
(2018-01-08 23:11:27.689663)     main()
(2018-01-08 23:11:27.689675)   File "05_get_client_ip_address", line 40, in main
(2018-01-08 23:11:27.689686)     result = get_ip_address('1.2.3.4', username, password)
(2018-01-08 23:11:27.689696)   File "05_get_client_ip_address", line 23, in get_ip_address
(2018-01-08 23:11:27.689706)     client = Client(host, username, password)
(2018-01-08 23:11:27.689780)   File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 242, in __init__
(2018-01-08 23:11:27.693277)     self.authenticate(username, password)
(2018-01-08 23:11:27.693306)   File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 250, in authenticate
(2018-01-08 23:11:27.693324)     return self.umc_auth(username, password)
(2018-01-08 23:11:27.693336)   File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 286, in umc_auth
(2018-01-08 23:11:27.693347)     return self.request('POST', 'auth', data)
(2018-01-08 23:11:27.693357)   File "/usr/lib/pymodules/python2.7/univention/testing/umc.py", line 69, in request
(2018-01-08 23:11:27.705862)     response = super(Client, self).request(method, path, data, headers)
(2018-01-08 23:11:27.705890)   File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 297, in request
(2018-01-08 23:11:27.705901)     return self.send(request)
(2018-01-08 23:11:27.705912)   File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 312, in send
(2018-01-08 23:11:27.705922)     raise ConnectionError('Could not send request.', reason=exc)
(2018-01-08 23:11:27.705933) univention.lib.umc.ConnectionError: ('Could not send request.', error(111, 'Connection refused'))
Comment 1 Florian Best univentionstaff 2018-01-10 15:10:52 CET
This causes that every following UMC test will also fail with: ('Could not send request.', error(111, 'Connection refused'))

I think the network redirection rules / iptables are responsible for this. I cannot reproduce this in my VM. Maybe it has to do with the network interfaces in the EC2 jenkins instances?
Comment 2 Florian Best univentionstaff 2018-01-10 15:39:41 CET
*** Bug 45937 has been marked as a duplicate of this bug. ***
Comment 3 Florian Best univentionstaff 2018-01-15 13:38:20 CET
0c4a80042bbe | Bug #46003: add temporary debug output
Comment 4 Florian Best univentionstaff 2018-01-16 18:41:39 CET
The debugging output shows that apache is not running. I think apache somehow crashes or gets stopped by some apache test script prior (Bug #45924)?
Comment 5 Florian Best univentionstaff 2018-01-19 09:55:57 CET
All the test were failing because apache was down. All tests are passing again since 2 days. I don't know what fixed it.
Comment 6 Stefan Gohmann univentionstaff 2018-02-09 16:05:35 CET
OK, it works again.
Comment 7 Stefan Gohmann univentionstaff 2018-03-14 14:38:08 CET
UCS 4.3 has been released:
 https://docs.software-univention.de/release-notes-4.3-0-en.html
 https://docs.software-univention.de/release-notes-4.3-0-de.html

If this error occurs again, please use "Clone This Bug".