Bug 45888 - Add Python 3 support for listener modules
Add Python 3 support for listener modules
Status: NEW
Product: UCS
Classification: Unclassified
Component: Listener (univention-directory-listener)
UCS 4.2
Other Linux
: P5 enhancement (vote)
: ---
Assigned To: UCS maintainers
UCS maintainers
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-12-18 16:30 CET by Nico Gulden
Modified: 2018-04-14 14:01 CEST (History)
4 users (show)

See Also:
What kind of report is it?: Feature Request
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?: Yes
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nico Gulden univentionstaff 2017-12-18 16:30:45 CET
Currently, listener modules have to be implented in Python 2. I received the feedback that it would great to support Python 3 with the Listener API. This would make the development of listener plugins for products depending on Python 3 a lot easier.
Comment 2 Philipp Hahn univentionstaff 2017-12-19 09:15:56 CET
As Lisener Modules are executed inside the Lisener itself, it currently is not possible to embed two versions of the Python interpreter C code.
Earliest time for a switch to Python3 is UCS-5 in 2019-2020
See Bug #44950 on how to use Celery as an intermediate queue, which can then also be handled by Python3.
Comment 3 Daniel Tröder univentionstaff 2017-12-19 09:44:30 CET
Please note that using Celery-workers will make the listener module asynchronous.

If that is OK, than I strongly suggest to use the new async API introduced in Bug #44786. It will take care of launching jobs etc.

It will also take care that even when running in parallel the order of transactions will be preserved per entryUUID. When running without parallelism that is no concern.

A customer told me, that if we'd make a listener asynchronous he'd need a tool to determine the replication state.
Comment 4 Niko Wenselowski 2017-12-19 17:33:07 CET
(In reply to Philipp Hahn from comment #2)
> See Bug #44950 on how to use Celery as an intermediate queue, which can then
> also be handled by Python3.
Unfortunately this ticket seems to be non-public. Is there any chance that the description will be made public?
Comment 5 Daniel Tröder univentionstaff 2017-12-20 07:51:39 CET
done
Comment 6 Daniel Tröder univentionstaff 2017-12-20 17:29:06 CET
Why not run two listener processes? One serving
* Py2 listener modules from /u/l/u-d-l/system and the other
* Py3 listener modules from /u/l/u-d-l3/system

This would allow us and others to gradually migrate Py2 code to Py3.

There would have to be some kind of locking on the transaction queue, to ensure that one listener doesn't get ahead of the other.
Comment 7 Florian Best univentionstaff 2017-12-20 17:36:45 CET
(In reply to Daniel Tröder from comment #5)
> done

Sorry, I will undo this because the comments there contain customer information/names. You could strip down the relevant information and add them here.
Comment 8 Arvid Requate univentionstaff 2017-12-20 21:05:45 CET
> Why not run two listener processes?

This though crossed my mind too. Drawback: if we run them in parallel we would probably have two caches. Only the currrent python2 one would continue to do replication.py and eventually feeding another notifier (on a DC backup).

a smaller solution might be to make a classic python2 listener wrapper that runs a python3 (or whatever) handler script, passing all data to it.