Bug 47582 - multiserver jenkins jobs throw misleading erros with SSLError CERTIFICATE_VERIFY_FAILED
multiserver jenkins jobs throw misleading erros with SSLError CERTIFICATE_VER...
Status: RESOLVED WONTFIX
Product: UCS@school
Classification: Unclassified
Component: ucs-test
UCS@school 4.2
Other Linux
: P5 normal (vote)
: ---
Assigned To: UCS@school maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-08-13 14:26 CEST by Daniel Tröder
Modified: 2020-07-10 12:42 CEST (History)
2 users (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Tröder univentionstaff 2018-08-13 14:26:04 CEST
[slave2031-multi-s4-all-components] 2018-08-13T00:07:26.166498	Traceback (most recent call last):
[slave2031-multi-s4-all-components] 2018-08-13T00:07:26.166498	  File "/root/schoolinstaller.py", line 137, in
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	 <module>
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	    status = client.umc_command('schoolinstaller/progress').result
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	  File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 272, in umc_command
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	    return self.request('POST', 'command/%s' % (path,), data, headers)
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	  File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 297, in request
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	    return self.send(request)
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	  File "/usr/lib/pymodules/python2.7/univention/lib/umc.py", line 312, in send
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	    raise ConnectionError('Could not send request.', reason=exc)
[slave2031-multi-s4-all-components] 2018-08-13T00:07:31.187264	ConnectionError: ('Could not send request.', SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)'))

4.2: http://jenkins.knut.univention.de:8080/job/UCSschool%204.2/job/UCSschool%204.2%20Multiserver/SambaVersion=s4-all-components/lastBuild/artifact/test/autotest-203-ucsschool-multiserver-s4.log
4.3: http://jenkins.knut.univention.de:8080/job/UCSschool-4.3/job/Install%20Multiserver/Config=s4,TestGroup=base1/lastBuild/artifact/test/autotest-203-ucsschool-multiserver-s4.log
Comment 1 Jürn Brodersen univentionstaff 2018-08-14 10:26:15 CEST
Afaik the jobs are not failing because of this error. See:
"$UCS_REPO/test/utils/schoolinstaller.py" grep for "failcount"
The installer tries 1200 times before giving up...

This is only a temporary problem during the installation. But I think we should find out why this is happening :) (And until then: a better error message would be nice)
Comment 2 Jürn Brodersen univentionstaff 2018-11-05 14:43:11 CET
This seems to have failed last weekend:

[slave2032-multi-s4] 2018-11-03T00:52:02.065020	ConnectionError: ('Could not send request.', SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'))
[slave2032-multi-s4] 2018-11-03T00:52:02.065020	. ERROR: 1200 failed attempts - comitting suicide

Due to the rejoin in the school installer the ucsCA is deleted and readded (in "univention-join" see setup_ssl)

I don't know why it failed completely this time. But the error seems to happen during the time the ucsCA doesn't exists on the server.

Note:
update-ca-certificates generates a file under /etc/ssl/certs/ca-certificates.crt which includes all CAs on the system (as well as updating all symlinks). Curl for example uses that file and has therefore no problem with the removal of the ucsCA (update-ca-certificates is only called after the ucsCA is readded)
Comment 3 Jürn Brodersen univentionstaff 2018-11-05 14:59:18 CET
Possible fix: Change the ssl context for the umc_client to use the concatenated file (/etc/ssl/certs/ca-certificates.crt).
But I'm not sure if that should be fixed in this script or the umc client lib. I would prefer it to be changed in the lib.
Comment 4 Michel Smidt 2020-07-10 12:42:27 CEST
This issue has been filed against UCS@school 4.2.

UCS@school 4.2 is out of maintenance and many UCS@school components have changed in later releases. Thus, this issue is now being closed.

If this issue still occurs in newer UCS@school versions, please reopen it and update the UCS@school version. In this case please provide detailed information on how this issue is affecting you.