Bug 48446 - Listener user invitation upon user creation - backend
Listener user invitation upon user creation - backend
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Self Service
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 4.4
Assigned To: Felix Botner
Erik Damrose
:
Depends on:
Blocks: 48632
  Show dependency treegraph
 
Reported: 2019-01-11 12:17 CET by Felix Botner
Modified: 2019-03-12 13:41 CET (History)
1 user (show)

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Botner univentionstaff 2019-01-11 12:17:41 CET
Idea: 

You create a user with a "PasswordRecoveryEmail" email and a random password and a listener module on the selfservice hosts generate a token and send a email for this account.
Comment 1 Felix Botner univentionstaff 2019-01-14 16:57:13 CET
comitted in : ucs44/selfservice
packages: deb [trusted=yes] http://10.200.18.180/debian/ ucs44selfservice main

Added univention-self-service-invitation (dependency of univention-self-service-master) with a listener plugin to detect "user invitations" (new UDM user with PasswordRecoveryEmail and pwdChangeNextLogin=1). The listener only creates files in /var/cache/univention-directory-listener/selfservice-invitation to trigger a invitation.

A separate daemon checks this directory and sends_tokens via the UMC module.

Listener and daemon can be disabled by umc/self-service/invitation/enabled=false

To send a token, a user has to be created with:

-> udm users/user create --set password=univention --set lastname=test \
  --set PasswordRecoveryEmail=root@localhost \
  --set pwdChangeNextLogin=1 --set username=test1
Comment 2 Felix Botner univentionstaff 2019-01-17 13:55:22 CET
Added a test: http://jenkins.knut.univention.de:8080/job/UCS-4.4/job/UCS-4.4-0/view/Branch%20Tests/job/branch%20test%20base%20singlemaster/9/testReport/

QA: please make sure the invitation is not triggered during the installation (initialization) and resync of the listener
Comment 3 Erik Damrose univentionstaff 2019-01-28 18:48:35 CET
OK: user with attribute PasswordRecoveryEmail and pwdChangeNextLogin triggers an email password token.
OK: listener writes to /var/cache/univention-directory-listener/selfservice-invitation
OK: systemd service listens monitors directory, sends mail to configured user via UMC call
OK: u-self-service-invitation service: automatic restart after server password change & failed UMC login. I think a traceback in the logfile is okay in that case.

Reopen: The univention-self-service-invitation service could be improved
- Time between checks is 5 seconds. Each cycle, a new umcclient object is generated, which results in a new UMC Session. Watch umc-{web-}server.log. On my testsystem the umc-web-server constantly uses 20% CPU while creating new and dropping old connections. -> Only create a UMC connection if there is something to do

Open question: how to handle the situation where an activation mail could not be sent, but the user has been removed. Currently, one has to remove the .send file from the cache directory manually to avoid a restart deadlock in the invitation service
Comment 4 Felix Botner univentionstaff 2019-01-30 13:57:40 CET
(In reply to Erik Damrose from comment #3)

> Reopen: The univention-self-service-invitation service could be improved
> - Time between checks is 5 seconds. Each cycle, a new umcclient object is
> generated, which results in a new UMC Session. Watch umc-{web-}server.log.
> On my testsystem the umc-web-server constantly uses 20% CPU while creating
> new and dropping old connections. -> Only create a UMC connection if there
> is something to do

umc_client initialization is now done once before the loop and reused is a token should be send.

> 
> Open question: how to handle the situation where an activation mail could
> not be sent, but the user has been removed. Currently, one has to remove the
> .send file from the cache directory manually to avoid a restart deadlock in
> the invitation service

The service now checks if the user exists and has a "univentionPasswordSelfServiceEmail", if not the trigger file is deleted.

530bc105f0dfd41880dfa3307257b0f475726db3 - univention-self-service
Comment 5 Erik Damrose univentionstaff 2019-02-04 12:52:11 CET
OK: Only create one UMC session
OK: UMC session is recreated on timeout after 10 minuten
OK: Logging to /var/log/univention/management-console-module-passwordreset.log, same as UMC module which is used to send mails
OK: No re-triggering of mails when resyncing listener module
Verified, merge to master will be done after frontend changes are done
Comment 6 Florian Best univentionstaff 2019-02-20 12:54:45 CET
Sorry, i have to reopen it. I added a lot of comments to gitlab:
https://git.knut.univention.de/univention/ucs/commit/9fe8f0674344b0020276cb3dbf22eb9d2110cc2e

The least is, that it doesn't work for users which username end with any of ".", "s", "e, "n" or "d".
Also just filtering for uid=* is insufficient. Why can't the file contain the dn of that object? Or at least iterate over all found objects instead of checking if len(objs) == 1.
Comment 7 Florian Best univentionstaff 2019-02-20 12:58:26 CET
And another thing: Is the service started with runit? Otherwise it will crash after the first exception and never be started again. There is no error handling at all in the daemon.
Comment 8 Felix Botner univentionstaff 2019-02-20 13:06:53 CET
(In reply to Florian Best from comment #7)
> And another thing: Is the service started with runit? Otherwise it will
> crash after the first exception and never be started again. There is no
> error handling at all in the daemon.

The service is started with systemd, and restarted after exit with an error code!=0
Comment 9 Felix Botner univentionstaff 2019-02-20 13:29:54 CET
(In reply to Florian Best from comment #6)
> Sorry, i have to reopen it. I added a lot of comments to gitlab:
> https://git.knut.univention.de/univention/ucs/commit/
> 9fe8f0674344b0020276cb3dbf22eb9d2110cc2e
> 
> The least is, that it doesn't work for users which username end with any of
> ".", "s", "e, "n" or "d".

this has been changed already

> Also just filtering for uid=* is insufficient. Why can't the file contain
> the dn of that object? Or at least iterate over all found objects instead of
> checking if len(objs) == 1.

uid should be enough for now, the listener triggers the invitation for new users only
Comment 10 Florian Best univentionstaff 2019-02-20 13:35:11 CET
(In reply to Felix Botner from comment #8)
> (In reply to Florian Best from comment #7)
> > And another thing: Is the service started with runit? Otherwise it will
> > crash after the first exception and never be started again. There is no
> > error handling at all in the daemon.
> 
> The service is started with systemd, and restarted after exit with an error
> code!=0

This is nice! But: If the invitation of a user now fails the script crashes and is restarted by systemd. After it starts again it tries to invite the same user again which again fails and cause the daemon to crash. This is an endless loop and prevents that users which don't have failure are invited. So if there is one user e.g. for which the self-service is not allowed this will block every other user. Should I create a new bug for this or reopen this bug?
Comment 11 Florian Best univentionstaff 2019-02-20 14:02:15 CET
Patch available in fbest/48446-fix-infinite-loop.
Comment 12 Felix Botner univentionstaff 2019-02-20 14:13:38 CET
(In reply to Florian Best from comment #11)
> Patch available in fbest/48446-fix-infinite-loop.

yes, that makes sense
44738888b7473743f26e77ee90212149e7ebcb9a - univention-self-service
Comment 13 Florian Best univentionstaff 2019-02-20 14:17:49 CET
(In reply to Felix Botner from comment #9)
> (In reply to Florian Best from comment #6)
> > Sorry, i have to reopen it. I added a lot of comments to gitlab:
> > https://git.knut.univention.de/univention/ucs/commit/
> > 9fe8f0674344b0020276cb3dbf22eb9d2110cc2e
> > 
> > The least is, that it doesn't work for users which username end with any of
> > ".", "s", "e, "n" or "d".
> 
> this has been changed already
Well, the change was also insufficient:
user = os.path.basename(filename).split('.send')[0]

→ e.g.
>>> 'bernie.senders.send'.split('.send')[0]
'bernie'

I adjusted this in the branch as well:
>>> 'bernie.senders.send'.rsplit('.send', 1)[0]
'bernie.senders'
Comment 14 Felix Botner univentionstaff 2019-02-20 14:33:05 CET
(In reply to Florian Best from comment #13)
> (In reply to Felix Botner from comment #9)
> > (In reply to Florian Best from comment #6)
> > > Sorry, i have to reopen it. I added a lot of comments to gitlab:
> > > https://git.knut.univention.de/univention/ucs/commit/
> > > 9fe8f0674344b0020276cb3dbf22eb9d2110cc2e
> > > 
> > > The least is, that it doesn't work for users which username end with any of
> > > ".", "s", "e, "n" or "d".
> > 
> > this has been changed already
> Well, the change was also insufficient:
> user = os.path.basename(filename).split('.send')[0]
> 
> → e.g.
> >>> 'bernie.senders.send'.split('.send')[0]
> 'bernie'
> 
> I adjusted this in the branch as well:
> >>> 'bernie.senders.send'.rsplit('.send', 1)[0]
> 'bernie.senders'

OK, b106ae64f407e6b90160ba991d2877f875fe0a3c
Comment 15 Florian Best univentionstaff 2019-02-20 18:15:05 CET
And by the way, if there are objects for which the registration fails there will be a massive memory leak:

Each UMC module process uses ~ 60MB RAM:

root@master3:~# free -m                                                                                                                                                                                                                                                          
              total        used        free      shared  buff/cache   available
Mem:           3955        2626         320          29        1008        1038
Swap:          2047        1362         685
root@master3:~# pkill -f '/usr/bin/python2.7.*/usr/sbin/univention-management-console-module.*passwordreset'                                                                                                                                                                     
root@master3:~# free -m
              total        used        free      shared  buff/cache   available
Mem:           3955         971        1967          29        1016        2692
Swap:          2047        1362         685

The module closes after 10 minutes / 15 minutes, but the script will always add new processes every 10 seconds.
Let's hope this will not happen on customer environments.
Comment 16 Erik Damrose univentionstaff 2019-02-22 13:35:25 CET
OK: changes to daemon
OK: invitation workflow still works
OK: ~ i adjusted the daemon timeout to 60 seconds to lessen the memory leak due to parallel modules running; d0ec06b0
I added a changelog entry in 6faf7dd

Verified
Comment 17 Florian Best univentionstaff 2019-03-12 13:41:03 CET
UCS 4.4 has been released:
 https://docs.software-univention.de/release-notes-4.4-0-en.html
 https://docs.software-univention.de/release-notes-4.4-0-de.html

If this error occurs again, please use "Clone This Bug".