Univention Bugzilla – Bug 45998
update pyopenssl and python-cryptography version in ucs-school-4.2 scope to match requirement of twisted
Last modified: 2018-05-02 18:18:04 CEST
The version of pyopenssl and python-cryptography in the ucs-school-4.2 scope does not match the expected version of twisted v16, but of twisted v14. This creates a traceback in the OPSI app. Update pyopenssl and python-cryptography.
Can you attach the traceback for searchability?
From https://help.univention.com/t/ucs-update-broke-opsi-app-python-cryptography-issue/7652 Jan 6 09:03:58 hostname opsiconfd[5904]: Traceback (most recent call last): Jan 6 09:03:58 hostname opsiconfd[5904]: File “/usr/bin/opsiconfd”, line 12, in Jan 6 09:03:58 hostname opsiconfd[5904]: from opsiconfd.opsiconfd import main Jan 6 09:03:58 hostname opsiconfd[5904]: File “/usr/lib/python2.7/dist-packages/opsiconfd/opsiconfd.py”, line 68, in Jan 6 09:03:58 hostname opsiconfd[5904]: from OPSI.Backend.BackendManager import BackendManager Jan 6 09:03:58 hostname opsiconfd[5904]: File “/usr/lib/python2.7/dist-packages/OPSI/Backend/BackendManager.py”, line 40, in Jan 6 09:03:58 hostname opsiconfd[5904]: from OPSI.Backend.Backend import (Backend, BackendModificationListener, Jan 6 09:03:58 hostname opsiconfd[5904]: File “/usr/lib/python2.7/dist-packages/OPSI/Backend/Backend.py”, line 47, in Jan 6 09:03:58 hostname opsiconfd[5904]: from twisted.conch.ssh import keys Jan 6 09:03:58 hostname opsiconfd[5904]: File “/usr/lib/python2.7/dist-packages/twisted/conch/ssh/keys.py”, line 25, in Jan 6 09:03:58 hostname opsiconfd[5904]: from cryptography.hazmat.primitives.asymmetric.utils import ( Jan 6 09:03:58 hostname opsiconfd[5904]: ImportError: No module named utils
All required source packages were imported from jessie-BP to the ucs-school-4.2 scope. As python-setuptools can currently not be build in our buildsystem (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879865), I copied it from 4.3-0. It is needed for building only. debhelper>=10 and dpkg>=1.18 can also currently not be build in our buildsystem, so as it is needed for building only, it was also copied. And xz-tools and tar... All these packages must *not* be copied to the appcenter repository! dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/all/*setuptools*_33.1.1-1_all.deb ucs_4.3-0/all/*pkg-resources*_33.1.1-1_all.deb ucs_4.3-0/all/debhelper_10.2.5_all.deb ucs_4.2-0-ucs-school-4.2/all dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/all/dpkg-dev_1.18.24_all.deb ucs_4.3-0/all/libdpkg-perl_1.18.24_all.deb ucs_4.2-0-ucs-school-4.2/all dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/amd64/libdpkg-dev_1.18.24_amd64.deb ucs_4.2-0-ucs-school-4.2/amd64/ dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/amd64/dpkg_1.18.24_amd64.deb ucs_4.3-0/amd64/tar_1.29b-1.1_amd64.deb ucs_4.2-0-ucs-school-4.2/amd64/ dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/i386/dpkg_1.18.24_i386.deb ucs_4.3-0/i386/tar_1.29b-1.1_i386.deb ucs_4.2-0-ucs-school-4.2/i386/ dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/amd64/liblzma5_5.2.2-1.2+b1_amd64.deb ucs_4.3-0/amd64/liblzma-dev_5.2.2-1.2+b1_amd64.deb ucs_4.2-0-ucs-school-4.2/amd64/ dtroeder@dimma:/var/univention/buildsystem2/apt$ cp ucs_4.3-0/i386/liblzma5_5.2.2-1.2+b1_i386.deb ucs_4.3-0/i386/liblzma-dev_5.2.2-1.2+b1_i386.deb ucs_4.2-0-ucs-school-4.2/i386/ dtroeder@dimma:/var/univention/buildsystem2/apt$ repo-apt-ftparchive --release ucs_4.2-0-ucs-school-4.2 ------ Then python-cryptography and its dependencies were build. Branch: ucs_4.2-0 Scope: ucs-school-4.2 six 1.10.0-3~bpo8+1A~4.2.0.201801091453 python-idna 2.0-3~bpo8+1A~4.2.0.201801101727 python-ipaddress 1.0.16-1~bpo8+1A~4.2.0.201801091458 strip-nondeterminism 0.034-1~bpo8+1A~4.2.0.201801091541 dh-autoreconf 12~bpo8+1A~4.2.0.201801091544 pycparser 2.17-2~bpo8+1A~4.2.0.201801091837 pyasn1 0.1.9-1~bpo8+1A~4.2.0.201801101729 dh-python 2.20170125~bpo8+1A~4.2.0.201801091843 python-cffi 1.9.1-2~bpo8+1A~4.2.0.201801091845 python-cryptography-vectors 1.7.1-1~bpo8+1A~4.2.0.201801101648 python-py 1.4.31-1~bpo8+1A~4.2.0.201801101707 python-hypothesis 3.4.2-2~bpo8+1A~4.2.0.201801101710 pytest 3.0.3-1~bpo8+1A~4.2.0.201801101712 python-cryptography 1.7.1-3~bpo8+1A~4.2.0.201801101730 pyopenssl 16.0.0-1~bpo8+1A~4.2.0.201801101752 With v16 code was moved from python-twisted-conch to python-twisted-core. Runtime requirements of python-twisted-conch & -core are fulfilled now with python-openssl v16.0 and python-cryptography v1.7, both from jessie-backports (and their respective dependencies). Test: ----- # python -c 'import cryptography.hazmat.primitives.asymmetric.utils' && echo OK OK # univention-app install opsi [..] # # systemctl status opsiconfd.service ● opsiconfd.service - Opsi Configuration Service Loaded: loaded (/lib/systemd/system/opsiconfd.service; enabled) Active: active (running) since Do 2018-01-11 09:02:16 CET; 26min ago Main PID: 10097 (opsiconfd) CGroup: /system.slice/opsiconfd.service └─10097 /usr/bin/python /usr/bin/opsiconfd -D # ps xa | grep opsiconfd 8945 ? S 0:00 /bin/bash +e /usr/bin/opsiconfd-guard daemon 10097 ? S 0:00 /usr/bin/python /usr/bin/opsiconfd -D
Today I was greeted with this traceback, when I opened the UMC: ------------------------------------------------------------------- Traceback (most recent call last): File "/usr/sbin/univention-management-console-web-server", line 235, in check_queue cls.dispatch(queuerequest) File "/usr/sbin/univention-management-console-web-server", line 250, in dispatch client = SessionClient(ip=queuerequest.ip) File "/usr/sbin/univention-management-console-web-server", line 126, in __init__ self.client = Client() File "/usr/lib/pymodules/python2.7/univention/management/console/protocol/client.py", line 101, in __init__ self.__crypto_context = SSL.Context(SSL.TLSv1_METHOD) File "/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 272, in __init__ self._problems.append(e) File "/usr/lib/python2.7/dist-packages/cffi/api.py", line 312, in gc ImportError: No module named gc_weakref ------------------------------------------------------------------- I could reproduce it on a 2nd server, after updating python-cffi. It went away after "service univention-management-console-web-server restart", as the univention-management-console-web-server was then using the new pyopenssl binaries. I suggest to add the "umc-web-server restart" to ucs-school-lib/debian/python-ucs-school.postinst (with a conditional for updating to the current version).
The binary packages that would be upgraded on a users system are: python-openssl python-cryptography python-cffi python-pyasn1 python-six
python-enum34, python-ipaddress and python-idna do not exist in jessie. python-pyasn1 would be upgraded from 0.1.7-1 to 0.1.9-1. The most critical change happens in python-cryptography 0.8, where a lot of classes/functions are moved between Python modules (see https://cryptography.io/en/latest/changelog/#v0-8).
(In reply to Daniel Tröder from comment #5) > The binary packages that would be upgraded on a users system are: > > python-openssl > python-cryptography > python-cffi > python-pyasn1 > python-six Where do you want to upgrade these packages? in UCS? in UCS@school? I think this might be problematic, we had even in the past multiple problems with python-psutil. If OPSI depends on a more recent Twisted version it should either use virtualenv or docker.
(In reply to Florian Best from comment #7) > (In reply to Daniel Tröder from comment #5) > > The binary packages that would be upgraded on a users system are: > > > > python-openssl > > python-cryptography > > python-cffi > > python-pyasn1 > > python-six > > Where do you want to upgrade these packages? in UCS? in UCS@school? > I think this might be problematic, we had even in the past multiple problems > with python-psutil. If OPSI depends on a more recent Twisted version it > should either use virtualenv or docker. opsi runs with Twisted versions ranging from ancient 10.something until recent 17.x and works fine with the 14.x provided through UCS. This is until someone installs UCS@school and updates his packages because the updated python-twisted-conch 16.x provided through the UCS@school repo does not work with python-cryptography provided through the core UCS repos. I fail to see why opsi should make changes if the break is introduced through UCS@school.
(In reply to Niko Wenselowski from comment #8) > opsi runs with Twisted versions ranging from ancient 10.something until > recent 17.x and works fine with the 14.x provided through UCS. > This is until someone installs UCS@school and updates his packages because > the updated python-twisted-conch 16.x provided through the UCS@school repo > does not work with python-cryptography provided through the core UCS repos. > > I fail to see why opsi should make changes if the break is introduced > through UCS@school. Oh yes! Then I think it might be good to consider if our UCS@school import could run in a virtualenv / docker environment instead of changing distribution packages.
This has nothing to do with the import. UCS@school does not use twisted. It was built as a build dependency and uploaded to the app center by accident. We are currently in the process of evaluating the best way how to deal with it: either downgrade twisted or upgrade python-openssl and python-cryptography.
As discussed: the newer version of python-twisted is removed from UCS@school 4.2 v7 and therefore the update of python-cryptography et al. is not required anymore. A new SDB article describes what has to be done to downgrade python-twisted: https://help.univention.com/t/resolving-opsi-installation-update-issues-on-ucs-school-systems/7705 @Daniel: Please check if there are changes that have to be reverted.
No commits were made to SVN/GIT. I removed the following source packages from the buildsystems scope: * enum34 * twisted * python-cryptography * pyopenssl * python-ipaddress * python-cffi * pyasn1 * python-hypothesis * strip-nondeterminism * dh-autoreconf * pycparser * dh-python * python-py I removed the following binary packages from the repo: * all/python-enum34* * */*twisted* * */python*-cryptography*.deb * all/python*-openssl*.deb * all/python-ipaddress_*.deb * */python*-cffi*.deb * all/python*-pyasn1*.deb * all/python*-hypothesis*.deb * all/*nondeterminism*.deb * all/dh-autoreconf_*.deb * all/python*-pycparser_*.deb * all/dh-python_*.deb * all/*python-py_1.4*.deb --- * all/*pkg-resources*.deb * all/debhelper_10.2.5_all.deb * rm */*dpkg*_1.18.24*.deb * rm */liblzma5_5.2.2-1.2+b1*.deb
OK: UCS@school 4.2 appcenter repo OK: UCS@school 4.3 appcenter repo ??: scope ucs-school-4.2 ??: scope ucs-school-4.3 OK: git/svn
(In reply to Sönke Schwardt-Krummrich from comment #13) > ??: scope ucs-school-4.2 > ??: scope ucs-school-4.3 OK: scope ucs-school-4.2 OK: scope ucs-school-4.3