Univention Bugzilla – Bug 49147
Migrate management/univention-directory-manager-modules to dh_python2
Last modified: 2019-11-13 19:17:58 CET
The package management/univention-directory-manager-modules should be migrated to use dh_python instead of python-support. Migration-Guide: https://hutten.knut.univention.de/mediawiki/index.php/Python_DebHelper_Migration
Especially take care of: listener/udm_extension.py:# pretty much all of the functions below mimic update-python-modules, additionally creating the __init__.py module file if nexessary listener/udm_extension.py:» » p = subprocess.Popen(['/usr/sbin/update-python-modules', '-p', pysupport_filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE) listener/udm_extension.py:» » ud.debug(ud.LISTENER, ud.ERROR, '%s: update-python-modules -p %s failed: %s.' % (name, pysupport_filename, e)) listener/udm_extension.py:» ud.debug(ud.LISTENER, ud.ERROR, '%s: update-python-modules -p %s failed: %s.' % (name, pysupport_filename, stderr)) listener/udm_extension.py:» » p = subprocess.Popen(['/usr/sbin/update-python-modules', '-p'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) listener/udm_extension.py:» » ud.debug(ud.LISTENER, ud.ERROR, '%s: update-python-modules -p failed: %s.' % (name, e)) listener/udm_extension.py:» ud.debug(ud.LISTENER, ud.ERROR, '%s: update-python-modules -p failed: %s.' % (name, stderr))
Depends on: services/univention-nfs/81univention-nfs-server.inst:if [ -f /usr/share/pyshared/univention/admin/handlers/policies/thinclient.py ] ; then management/univention-ldap/univention-backup2master:if [ -f /usr/share/pyshared/univention/admin/handlers/policies/thinclient.py ] ; then
Depends on: doc/developer-reference/udm/udm.xml:» » » » » As an alternative the file can be put into the directory <filename class="directory">/usr/share/pyshared/univention/admin/syntax.d/</filename> and linked into the directory <filename class="directory">/usr/lib/pymodul
test/ucs-test/tests/68_udm-extendedattribute/26_extented_attribute_ldap_addlist_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/26_extented_attribute_ldap_addlist_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/27_extented_attribute_ldap_pre_create_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/27_extented_attribute_ldap_pre_create_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/28_extented_attribute_open_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/28_extented_attribute_open_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/29_extented_attribute_ldap_post_create_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/29_extented_attribute_ldap_post_create_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/30_extented_attribute_ldap_post_modify_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/30_extented_attribute_ldap_post_modify_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/31_extented_attribute_ldap_modlist_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/31_extented_attribute_ldap_modlist_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/32_extented_attribute_ldap_pre_modify_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/32_extented_attribute_ldap_pre_modify_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/33_extented_attribute_ldap_pre_remove_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/33_extented_attribute_ldap_pre_remove_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/34_extented_attribute_ldap_post_remove_hook:» atexit.register(os.remove, '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name) test/ucs-test/tests/68_udm-extendedattribute/34_extented_attribute_ldap_post_remove_hook:» with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: test/ucs-test/tests/68_udm-extendedattribute/40_extended_attribute_attributehook_value_mapping:» fn_hook = '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/{}.py'.format(hook_name)
Patch in branch dh-python/49147-univention-directory-manager-modules. WIP: listener/udm_extension.py
The branch without the changes for the listener/udm_extension.py has been build: univention-directory-manager-modules (14.0.14-8) f7413794994d | Bug #49147: Merge branch 'dh-python/49147-univention-directory-manager-modules' into 4.4-2 7d3146c0ac8b | Bug #49147: futurize scripts 3a4f34eb6273 | Bug #49147: pep8: scripts a4c8bfc3c7ce | Bug #49147: add dh-python integration 2b18eeb7b39d | Bug #49147: fix further python3 syntax errors ee8b4f97a472 | Bug #49147: libfuturize.fixes.fix_raise a744b7f97c96 | Bug #49147: libfuturize.fixes.fix_print_with_import a383115a42cc | Bug #49147: add python3-univention-directory-manager bc237fccee25 | Bug #49147: extend __path__ of UDM handler packages 4064463f58b7 | Bug #49147: adjust pyshared path in long description cfe67c31a17a | Bug #49147: import all syntax.d and hooks.d files 26c7c6cb59ae | Bug #49147: migrate to dh-python univention-directory-manager-rest (9.0.16-5) f7413794994d | Bug #49147: Merge branch 'dh-python/49147-univention-directory-manager-modules' into 4.4-2 a4c8bfc3c7ce | Bug #49147: add dh-python integration univention-directory-manager-rest.yaml f7413794994d | Bug #49147: Merge branch 'dh-python/49147-univention-directory-manager-modules' into 4.4-2 fdd77b08f9de | YAML Bug #49147 univention-directory-manager-modules.yaml f7413794994d | Bug #49147: Merge branch 'dh-python/49147-univention-directory-manager-modules' into 4.4-2 fdd77b08f9de | YAML Bug #49147 Unfortionately the Jenkins tests for registering ldap extensions and some extended attribute tests fail: https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-2/job/AutotestJoin/44/SambaVersion=s4,Systemrolle=master/testReport/72_udm-extensions/01_register_deregister_via_joinscript/master091/ https://jenkins.knut.univention.de:8181/job/UCS-4.4/job/UCS-4.4-2/job/AutotestJoin/44/SambaVersion=s4,Systemrolle=master/testReport/68_udm-extendedattribute/26_extented_attribute_ldap_addlist_hook/master091/
The interim problem was that the new package did not contain some directories anymore: /usr/share/pyshared/univention/admin/hooks.d/ But the old listener/udm_extension.py still expects them to exists. listener.log: 23.10.19 05:49:11.085 LISTENER ( PROCESS ) : updating 'cn=udm_hook,cn=univention,dc=AutoTest091,dc=local' command a 23.10.19 05:49:11.267 LISTENER ( PROCESS ) : updating 'cn=p6uzgcbleo,cn=udm_hook,cn=univention,dc=AutoTest091,dc=local' command a 23.10.19 05:49:11.268 LISTENER ( ERROR ) : udm_extension: Writing new data to /usr/share/pyshared/univention/admin/hooks.d/p6uzgcbleo.py failed: [Errno 2] No such file or directory: '/usr/share/pyshared/univention/admin/hooks.d/p6uzgcbleo.py'. extended attribute test-case: (2019-10-23 05:17:12.331405) Traceback (most recent call last): (2019-10-23 05:17:12.331505) File "26_extented_attribute_ldap_addlist_hook", line 22, in <module> (2019-10-23 05:17:12.331582) with open('/usr/lib/pymodules/python2.7/univention/admin/hooks.d/%s.py' % hook_name, 'w') as hook_module: (2019-10-23 05:17:12.331635) IOError: [Errno 2] No such file or directory: '/usr/lib/pymodules/python2.7/univention/admin/hooks.d/ld7gnc92ng.py' Re-added those directories: univention-directory-manager-modules (14.0.14-9) 369b982d7c59 | Bug #49147: re-add missing pyshared pathes Everything else which needs to be done e.g. regarding the listner (comment 1-4) has been split into Bug #50401.
The listener tests are working again. But still some extended attribute tests were failing. Somehow the old pyshared/pymodules directories are removed altough they are part of the debian package. Maybe the python-support logic is responsible for this. As workaround I just add empty __init__.py files. univention-directory-manager-modules (14.0.14-11) 3d0732622c70 | Bug #49147: re-add missing pyshared pathes
What is removing the files and folders??? # ls /usr/lib/pymodules/python2.7/univention/admin/ handlers __init__.py __init__.pyc syntax.d → missing hooks.d # dpkg -L python-univention-directory-manager | grep hooks.d /usr/lib/pymodules/python2.7/univention/admin/hooks.d /usr/lib/pymodules/python2.7/univention/admin/hooks.d/__init__.py /usr/lib/python2.7/dist-packages/univention/admin/hooks.d /usr/lib/python2.7/dist-packages/univention/admin/hooks.d/__init__.py /usr/share/pyshared/univention/admin/hooks.d /usr/share/pyshared/univention/admin/hooks.d/__init__.py → but they are part of the package.
Aha! # ls /usr/lib/pymodules/python2.7/univention/admin/ handlers hooks.d __init__.py syntax.d # update-python-modules -a -f # ls /usr/lib/pymodules/python2.7/univention/admin/ handlers __init__.py __init__.pyc syntax.d
Fixed by re adding python-support to the package for those leacy files. univention-directory-manager-modules (14.0.14-13) 5dd6714ad40d | Bug #49147: remove directories from .public file a4f1310cf1d0 | Bug #49147: use python-support for legacy files
(In reply to Florian Best from comment #9) > What is removing the files and folders??? "/usr/lib/pymodules/" is owned by python-support and it can remove/modify anything below that directory on its own whim! No package is supposed to install files there directly, but only indirectly through "/usr/share/pyshared/" and an appropriate control-file "/usr/share/python-support/$pkg.public" for `update-python-modules`.
I had to adjust version numbers in the control file, due to an error during the upgrade: ''' dpkg: error processing archive /tmp/apt-dpkg-install-ez4Cx5/05-python-univention-directory-manager_14.0.14-15A~4.4.0.201911041204_all.deb (--unpack): trying to overwrite '/usr/lib/python2.7/dist-packages/univention/admin/__init__.py', which is also in package python-univention-directory-manager-rest 9.0.16-4A~4.4.0.201909231600 dpkg-deb: error: subprocess paste was killed by signal (Broken pipe) ''' [4.4-2 f26f7d9407] Bug #49147: fix version in dependencies [4.4-2 3ee968c203] Bug #49147: yaml What I tested: Update -> OK UMC login > OK Add user -> OK udm saml/serviceprovider list -> OK jenkins -> OK yaml -> OK -> verified
You changed the dependencies without the debian/changelog? I don't understand how this is working?
(In reply to Florian Best from comment #14) > You changed the dependencies without the debian/changelog? I don't > understand how this is working? I removed the source revision, imported the revision and rebuild the package.
[FAIL] changes.valid: Mismatching binary package version: 9.0.16-5A~4.4.0.201911041221 != python-univention-directory-manager-rest 9.0.16-6A~4.4.0.201911070937 from univention-directory-manager-rest 9.0.16-6A~4.4.0.201911070937
A broken python3- dependency has been removed from python3-univention-directory-manager-rest. univention-directory-manager-rest (9.0.16-6) 377a002ea65e | Bug #49147: remove unnecessary dependency
(In reply to Florian Best from comment #17) > A broken python3- dependency has been removed from > python3-univention-directory-manager-rest. > > univention-directory-manager-rest (9.0.16-6) > 377a002ea65e | Bug #49147: remove unnecessary dependency ErrataValidation -> OK
<http://errata.software-univention.de/ucs/4.4/347.html> <http://errata.software-univention.de/ucs/4.4/348.html>