Univention Bugzilla – Bug 54192
Python 3 migration errors in OX app
Last modified: 2022-10-26 11:23:43 CEST
See https://help.univention.com/t/upgrade-auf-ucs-5-ox/18871/8 ----------------------------------------------------------------------- raceback (most recent call last): File "/usr/lib/univention-directory-listener/system/ox-user.py", line 1119, in <module> handler.process_pending() File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 720, in process_pending return self.process(func) File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 764, in process entry = self._load_entry(timestamp) File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 829, in _load_entry return pickle.load(fd) UnicodeDecodeError: 'ascii' codec can't decode byte 0xa1 in position 2: ordinal not in range(128) ----------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/univention-directory-listener/system/ox-resource.py", line 136, in <module> handler.process_pending() File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 720, in process_pending return self.process(func) File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 775, in process result = func(dn, entry.new, entry.old, entry.action) File "/usr/lib/univention-directory-listener/system/ox-resource.py", line 132, in handler ox_resource_listener = OXResourceListener(name, old, new) File "/usr/lib/univention-directory-listener/system/ox-resource.py", line 52, in __init__ self.resource_name = self.get_resource_name() File "/usr/lib/univention-directory-listener/system/ox-resource.py", line 59, in get_resource_name return self.new['cn'][0].decode('utf-8') AttributeError: 'str' object has no attribute 'decode' ----------------------------------------------------------------------- /usr/share/univention-ox/manage_listener_queue --list Traceback (most recent call last): File "/usr/share/univention-ox/manage_listener_queue", line 266, in <module> sys.exit(main()) File "/usr/share/univention-ox/manage_listener_queue", line 220, in main handler.collect_pickle_files() File "/usr/share/univention-ox/manage_listener_queue", line 160, in collect_pickle_files dn_s TypeError: unsupported format string passed to bytes.__format__
Issue: See listed tracebacks by Daniel Tröder Fix: corrected code issues according to new python3 standards fixed with branch: asteffen/54192-python3-migration-errors requesting QA
QA: Code review: I have commented some remarks here https://git.knut.univention.de/univention/open-xchange/appsuite/-/commit/b3316c62bac8cb46d5f7ea644a01c1ced9fa2191
QA The commit messages need a "#" Bug 54192 -> Bug #54192 (I think this was a misunderstanding) please re-add the comment: # FIXME: default_context is int! File "/usr/share/univention-ox/manage_listener_queue", line 266, in <module> sys.exit(main()) File "/usr/share/univention-ox/manage_listener_queue", line 220, in main handler.collect_pickle_files() File "/usr/share/univention-ox/manage_listener_queue", line 156, in collect_pickle_files ox_module.rjsut(8), AttributeError: 'str' object has no attribute 'rjsut' -> after your fix, if resolve the bug, so the QA knows it's time to continue.
fixed typo in manage_listener_queue, readded comment in ox-resource.py https://git.knut.univention.de/univention/open-xchange/appsuite/-/commit/78f1a4af83c33d45ed5db4caa1f70cfdae6fea2b
QA: looks good. reopen for final steps. could you please explain again, why replaced '{} | {:8} | {} | {:18} | {}{}' with "%s | %s | %s | %s | %s%s" - Don't forget the `#` in the commit message and to change to something like "fix python3 migration errors". When you do your squashing, please add the spaces (revert) to keep the git diff small #FIXME: default_context is int! After that you can squash, merge & build. Ask me, if you need help with that.
All requestes changes by Tobias Wenzel where added. Cherry-picked and merged the branch into 5.0 Successful build Package: univention-ox Version: 12.0.25A~5.0.0.202201180917 Branch: ucs_5.0-0 Scope: oxse4ucs
QA: all OK -> verify OK code OK changelog OK merge OK script is working
REOPEN: univention-ox/listener/ox-resource.py * get_resource_name() returns mixed bytes/str * self.old_context_id and self.new_context_id is assigned to different types (bytes vs str) - depends on if the attribute is set or not * `__package__ = '' # workaround for PEP 366` was not removed, like explained in the Dev-Doc. * self.resource_name is of type bytes but this is given to string formatting in remove_resource() univention-ox/share/manage_listener_queue: * collect_pickle_files(): a unnecessary f-string has been commited, which is also not Python 2 compatible (don't know if the app is also released for py2). * collect_pickle_files(): looks like a byte string (variable "name") is mixed into options_str. I say looks like because I don't know if the other OX-classes are changing old and new to contain strings instead of bytestrings - which if yes would probably be wrong as the encoding depends on the specific attribute.
I am thinking to increase the pain, because the "rejected" user cannot use/login in the appsuite. Ein Benutzer mit der ID "usupport" konnte nicht im Kontext 10 gefunden werden. (USR-0015) I see in the listener.log 08.02.22 12:49:44.237 LISTENER ( PROCESS ) : updating 'uid=usupport,ou=New,ou=Mitarbeiter,ou=Neu,dc=schein,dc=de' command a Traceback (most recent call last): File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 728, in wrapper return self.process(func) File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 764, in process entry = self._load_entry(timestamp) File "/usr/lib/python3/dist-packages/univention/ox/listener_tools.py", line 829, in _load_entry return pickle.load(fd) UnicodeDecodeError: 'ascii' codec can't decode byte 0xa1 in position 2: ordinal not in range(128) The listener then tries to add the user 3 more time with the same traceback
univention-app info UCS: 5.0-1 errata218 Installed: mailserver=12.0 open-xchange-guard=2.10.5-ucs1 open-xchange-text=7.10.5-ucs1 oxseforucs=7.10.5-ucs4 samba4=4.13 Upgradable: apt-cache policy open-xchange open-xchange: Installiert: 7.10.5-38 Sorry forgot
fixes: fixed mixed return-types and removed the workaround for PEP 366 which is not needed anymore. fixed with branch: 54192-python3-migration-errors-v2 requesting QA @Christina Scheinig: with the implemented fixes, I was not able to reproduce the behaviour anymore.
QA: looks good. please squash, merge & build.
Successful build Package: univention-ox Version: 12.0.27A~5.0.0.202203040936 Branch: ucs_5.0-0 Scope: oxse4ucs
QA: All OK verified
When is a new appversion with this fix scheduled?
I increased the pain again, because new user cannot use the appsuite, so e-mail workaround is thunderbird but no calender and addressbook for the user.
I'm very sorry. This should have been released with 7.10.5-ucs5, but (I guess) it was missed due to a missing entry in the update notes. It will be released with the next version - sorry, I can't tell you a date.
(In reply to Tobias Wenzel from comment #17) > I'm very sorry. This should have been released with 7.10.5-ucs5, but (I > guess) it was missed due to a missing entry in the update notes. > > It will be released with the next version - sorry, I can't tell you a date. REOPEN: Then please add the entry into the update notes.
and please set a Target Milestone for every bug which you RESOLVE/VERIFY.
added target milestone, update notes have been adapted in [5.0] aa699768 Bug #54192: add update entry
Could you please provide a patch, the customer can apply, so the new users can work with OX?
Created attachment 10954 [details] ox-resource.py patch
Created attachment 10955 [details] manage_listerner_queue patch
Copy the patches to your UCS system apply the patches like so: (In reply to Tobias Wenzel from comment #22) > Created attachment 10954 [details] > ox-resource.py patch patch /usr/share/univention-ox/listener/ox-resource.py < /tmp/ox-resource.py.patch (In reply to Tobias Wenzel from comment #22) > Created attachment 10955 [details] > manage_listerner_queue patch patch /usr/share/univention-ox/manage_listener_queue /tmp/manage_listener_queue.patch
Created attachment 10956 [details] patch for listener_tools.py patch /usr/lib/python3/dist-packages/univention/ox/listener_tools.py < /tmp/listener_tools.py
set to resolve/ moved: i added changes for the listener_tools.py in https://git.knut.univention.de/univention/open-xchange/appsuite/-/merge_requests/3/diffs#ac04d381579ac0ae332ce6f713fc73d1124f9938 this has to be qa't as well.
Package: univention-ox Version: 12.0.28A~5.0.0.202205240829 Branch: ucs_5.0-0 Scope: oxse4ucs [5.0] 0d617637 Bug #54192: migrate listener_tools.py changelog: Bug #54192: python3 migration: use byte encoding to load pickle file
I now, have the third customer, with a support ticket, who is not able to apply the patches!
Released with OX app version 7.10.6-ucs1.