Univention Bugzilla – Bug 49325
cherry-pick python3-ldap from buster
Last modified: 2019-12-06 00:03:05 CET
We need to cherry-pick python3-ldap from buster: https://packages.debian.org/buster/python3-ldap +++ This bug was initially created as a clone of Bug #49060 +++
Completely unrelated, but maybe of interest nonetheless: I have successfully used the pure-Python LDAP library with UCS: https://github.com/cannatag/ldap3 https://ldap3.readthedocs.io/ It has a very nice API and it support paging! search_paged() returns a Python iterator :) If UDM and UMC were to use that too, UCS could gain a lot of performance on large datasets.
Using the version from buster succeeds when patching debian/compat to 10, remove debhelper=11 build dependency, remove python3-pyasn1 version dependency in setup.py. dpkg: dependency problems prevent configuration of python3-ldap:amd64: python3-ldap:amd64 depends on python3-pyasn1 (>= 0.3.7); however: Version of python3-pyasn1 on system is 0.1.9-2. So probably a good idea to also update the python3-pyasn1 package: https://packages.debian.org/buster/python3-pyasn1
(In reply to Florian Best from comment #2) > Using the version from buster succeeds when patching debian/compat to 10, > remove debhelper=11 build dependency, remove python3-pyasn1 version > dependency in setup.py. > > dpkg: dependency problems prevent configuration of python3-ldap:amd64: > python3-ldap:amd64 depends on python3-pyasn1 (>= 0.3.7); however: > Version of python3-pyasn1 on system is 0.1.9-2. > > So probably a good idea to also update the python3-pyasn1 package: > https://packages.debian.org/buster/python3-pyasn1 There was in incompatible API change in pyasn1 which lead to a lengthy discussion on python-ldap starting 2017-11-07. This might be okay if there is no other user in UCS except python-ldap. Also: how will you handle security monitoring for packages picked from Debian-Buster? The Security team will not do that for you by default!
(In reply to Philipp Hahn from comment #3) > Also: how will you handle security monitoring for packages picked from > Debian-Buster? The Security team will not do that for you by default! The security team has to! We had those cases in the past already and Arvid added tracking for it (e.g. python-saml2, simplesamlphp, etc.). Or do you have an alternative to cherry-picking those packages?!
> We had those cases in the past already and Arvid added tracking for it (e.g. python-saml2, simplesamlphp, etc.). I had a tracking in my brain. Now things are imported automatically and that probably has not been implemented yet. But I agree, we need that.
> There was in incompatible API change in pyasn1 which lead to a lengthy discussion on python-ldap starting 2017-11-07. This might be okay if there is no other user in UCS except python-ldap. univention-management-console-module-diagnostic/umc/python/diagnostic/plugins/22_kdc_service.py is another candiate.
# apt-cache rdepends python-pyasn1 python3-pyasn1 | sort | uniq python3-cryptography python3-ndg-httpsclient python3-paramiko python3-pyasn1-modules python3-pysnmp4 python3-requests python3-service-identity python3-sleekxmpp python3-urllib3 python-cryptography python-ldap python-ndg-httpsclient python-paramiko python-pyasn1-modules python-pysaml2 python-pysnmp4 python-requests python-service-identity python-sleekxmpp python-twisted-conch python-urllib3 univention-management-console-module-diagnostic That doesn't sound good/easy. We are using paramiko, pysaml2, ldap, requests, (cryptography, urllib3).
I just noticed that the (now merged) fork of python-ldap for python3 is in stretch : https://packages.debian.org/stretch/python3-pyldap Can we use that? A quick look at the diff of the current python-ldap and python3-pyldap packages in debian stretch doesn't show any crazy api changes between the two.
(In reply to Jürn Brodersen from comment #8) > I just noticed that the (now merged) fork of python-ldap for python3 is in > stretch : > > https://packages.debian.org/stretch/python3-pyldap > > Can we use that? A quick look at the diff of the current python-ldap and > python3-pyldap packages in debian stretch doesn't show any crazy api changes > between the two. python3-pyldap is the project I mentioned in comment1. It is a pure-Python LDAP API and is not API compatible.
(In reply to Daniel Tröder from comment #9) > (In reply to Jürn Brodersen from comment #8) > > I just noticed that the (now merged) fork of python-ldap for python3 is in > > stretch : > > > > https://packages.debian.org/stretch/python3-pyldap > > > > Can we use that? A quick look at the diff of the current python-ldap and > > python3-pyldap packages in debian stretch doesn't show any crazy api changes > > between the two. > python3-pyldap is the project I mentioned in comment1. It is a pure-Python > LDAP API and is not API compatible. No, you are speaking about python3-ldap3 while Jürn speaks about python3-pyldap (which is a fork of python-ldap). Maybe we could start with a dependency like: Depends: python3-pyldap | python3-ldap.
Note: In buster https://packages.debian.org/buster/python3-pyldap the package description says "This transition dummy package for Python 3.x can be safely removed, since the ldap module has merged changes in the pyldap fork." -- weird stuff.
OK: for now we can depend on python3-pyldap, I think.
With the python-ldap package from Debian Buster I experience a lot of exceptions, e.g.: 18.10.2019 16:00:38.909 LDAP (INFO ): samaccount_dn_mapping: search in s4 for (&(objectclass=group)(samaccountname=foogroup1)) 18.10.2019 16:00:38.911 LDAP (WARNING): sync failed, saved as rejected /var/lib/univention-connector/s4/1571407238.881395 18.10.2019 16:00:38.912 LDAP (WARNING): Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 1113, in poll_ucs sync_successfull = self.__sync_file_from_ucs(filename, traceback_level=traceback_level) File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 887, in __sync_file_from_ucs mapped_object = self._object_mapping(key, object, 'ucs') File "/usr/lib/python2.7/dist-packages/univention/s4connector/__init__.py", line 1838, in _object_mapping object = function(self, object, dn_mapping_stored, isUCSobject=(object_type == 'ucs')) File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/__init__.py", line 459, in group_dn_mapping return samaccountname_dn_mapping(s4connector, given_object, dn_mapping_stored, isUCSobject, 'group', u'cn', u'posixGroup', 'cn', u'group') File "/usr/lib/python2.7/dist-packages/univention/s4connector/s4/__init__.py", line 384, in samaccountname_dn_mapping newdn = unicode(ldap.dn.dn2str([newdn_rdn] + exploded_dn[1:]), 'utf8') # new object, don't need to change TypeError: decoding Unicode is not supported
Also experience this traceback with the version from buster: UNIVENTION_DEBUG_BEGIN : uldap.__open host=master89.school.dev port=7389 base=l=school,l=dev Traceback (most recent call last): File "/usr/lib/univention-directory-listener/system/pkgdb-watch.py", line 87, in setdata ldap_reconnect() File "/usr/lib/univention-directory-listener/system/pkgdb-watch.py", line 65, in ldap_reconnect ldap_info['lo'] = univention.uldap.access(host=ldap_info['ldapserver'], base=ldap_info['basedn'], binddn=ldap_info['binddn'], bindpw=ldap_info['bindpw'], start_tls=2) File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 275, in __init__ self.__open(ca_certfile) File "/usr/lib/python2.7/dist-packages/univention/uldap.py", line 344, in __open self.lo = ldap.ldapobject.ReconnectLDAPObject(self.uri, trace_stack_limit=None, retry_max=self.client_connection_attempt, retry_delay=1) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 1098, in __init__ bytes_strictness=bytes_strictness) File "/usr/lib/python2.7/dist-packages/ldap/ldapobject.py", line 99, in __init__ self._trace_level = trace_level or ldap._trace_level AttributeError: 'module' object has no attribute '_trace_level'
Maybe the bytes_mode=True/False helps us: https://www.python-ldap.org/en/python-ldap-3.2.0/bytes_mode.html