Bug 51366 - socket leak in pam handling
socket leak in pam handling
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.4
Other Linux
: P5 normal (vote)
: UCS 4.4-4-errata
Assigned To: Florian Best
Felix Botner
:
Depends on:
Blocks: 51047
  Show dependency treegraph
 
Reported: 2020-05-28 11:34 CEST by Felix Botner
Modified: 2020-06-19 17:32 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 4: Minor Usability: Impairs usability in secondary scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains
How will those affected feel about the bug?: 2: A Pain – users won’t like this once they notice it
User Pain: 0.137
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments
repr1.sh (550 bytes, application/x-shellscript)
2020-05-28 11:35 CEST, Felix Botner
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Botner univentionstaff 2020-05-28 11:34:08 CEST
see Bug #51047

In our pam handling in pam.py and auth.py we do not properly remove the python-pam object, pam_end is not called and resources (e.g. ldap connections from pam_ldap) not "freed".


We already have a patch


--- a/management/univention-management-console/src/univention/management/console/auth.py
+++ b/management/univention-management-console/src/univention/management/console/auth.py
@@ -128,6 +128,8 @@ class AuthHandler(signals.Provider):
                else:
                        AUTH.info('Authentication for %r was successful' % (username,))
                        return (username, password)
+               finally:
+                       self.pam.pam = None
 
        def __canonicalize_username(self, username):
                try:

But in my test the following problem occurred:

seems that the self.pam.pam = None in __authenticate_thread() (or __authentication_result(), tested both) is not "unproblematic"

"Sometimes" when i login via UMC /univention/management/ and after the login i restart the umc server, i get the following traceback:

umc-server.log
20.05.20 22:40:12.635  MAIN        ( PROCESS ) : Server started
20.05.20 22:40:21.596  MODULE      ( PROCESS ) : Setting auth type to None
20.05.20 22:40:21.764  MAIN        ( PROCESS ) : Updating user password in 0 running module processes (auth-type: None).
20.05.20 22:40:23.795  MAIN        ( PROCESS ) : LDAP bind for user 'uid=Administrator,cn=users,dc=four,dc=four'.
20.05.20 22:40:24.891  MAIN        ( PROCESS ) : running: ['/usr/bin/python2.7', '/usr/sbin/univention-management-console-module', '-m', 'join', '-s', '/var/run/univention-management-console/25003-1590007224891.socket', '-d', '3', '-l', 'en_US.UTF-8']
20.05.20 22:40:25.139  MAIN        ( PROCESS ) : running: ['/usr/bin/python2.7', '/usr/sbin/univention-management-console-module', '-m', 'updater', '-s', '/var/run/univention-management-console/25003-1590007225138.socket', '-d', '3', '-l', 'en_US.UTF-8']
20.05.20 22:40:25.259  MAIN        ( PROCESS ) : running: ['/usr/bin/python2.7', '/usr/sbin/univention-management-console-module', '-m', 'appcenter', '-s', '/var/run/univention-management-console/25003-1590007225259.socket', '-d', '3', '-l', 'en_US.UTF-8']
20.05.20 22:40:25.389  MAIN        ( PROCESS ) : running: ['/usr/bin/python2.7', '/usr/sbin/univention-management-console-module', '-m', 'udm', '-s', '/var/run/univention-management-console/25003-1590007225388.socket', '-d', '3', '-l', 'en_US.UTF-8']
20.05.20 22:40:30.495  DEBUG_INIT
20.05.20 22:40:30.498  MAIN        ( WARN    ) : Shutting down all open connections
20.05.20 22:40:31.037  DEBUG_INIT  
20.05.20 22:40:32.312  MAIN        ( PROCESS ) : Server started
20.05.20 22:40:37.850  MODULE      ( PROCESS ) : Setting auth type to None
20.05.20 22:40:38.020  MAIN        ( PROCESS ) : Updating user password in 0 running module processes (auth-type: None).
20.05.20 22:40:38.023  AUTH        ( ERROR   ) :   File "/usr/lib/python2.7/dist-packages/notifier/threads.py", line 80, in _run
    result = self._function()      
  File "/usr/lib/python2.7/dist-packages/notifier/__init__.py", line 104, in __call__
    return self._function(*tmp, **self._kwargs)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/auth.py", line 111, in __authenticate_thread
    self.pam.authenticate(username, password, **custom_prompts)
  File "/usr/lib/python2.7/dist-packages/univention/management/console/pam.py", line 213, in authenticate
    self.start(username, (answers, [], missing))
  File "/usr/lib/python2.7/dist-packages/univention/management/console/pam.py", line 258, in start
    self.pam.set_item(PAM_CONV, self.conversation)
AttributeError: 'NoneType' object has no attribute 'set_item'

20.05.20 22:40:38.089  MODULE      ( PROCESS ) : Setting auth type to None
20.05.20 22:40:38.266  MAIN        ( PROCESS ) : Updating user password in 0 running module processes (auth-type: None).
20.05.20 22:40:38.273  MODULE      ( PROCESS ) : Setting auth type to None 
20.05.20 22:40:38.528  MAIN        ( PROCESS ) : Updating user password in 0 running module processes (auth-type: None).
20.05.20 22:40:38.640  MODULE      ( PROCESS ) : Setting auth type to None 
20.05.20 22:40:38.895  MAIN        ( PROCESS ) : Updating user password in 0 running module processes (auth-type: None).
20.05.20 22:40:38.952  MAIN        ( PROCESS ) : running: ['/usr/bin/python2.7', '/usr/sbin/univention-management-console-module', '-m', 'udm', '-s', '/var/run/univention-management-console/25179-1590007238952.socket', '-d', '3', '-l', 'en_US.UTF-8']
20.05.20 22:40:38.990  MODULE      ( PROCESS ) : Setting auth type to None
20.05.20 22:40:39.381  MAIN        ( PROCESS ) : Updating user password in 1 running module processes (auth-type: None).
20.05.20 22:40:39.394  LOCALE      ( WARN    ) : Could not find translation file: 'umc-core.mo'
20.05.20 22:40:39.394  MAIN        ( WARN    ) : Module None (command=u'udm/reports/query', id=u'159000723765067-1171') does not exists anymore
20.05.20 22:40:39.449  MODULE      ( PROCESS ) : Verboten
20.05.20 22:40:39.456  MODULE      ( PROCESS ) : For using this request a login is required.
20.05.20 22:40:39.458  MODULE      ( PROCESS ) : For using this request a login is required.
20.05.20 22:40:39.458  MAIN        ( WARN    ) : Module None (command=u'udm/containers', id=u'159000723764337-1167') does not exists anymore
20.05.20 22:40:39.523  MODULE      ( PROCESS ) : Verboten

umc-web-server.log:
20.05.20 22:39:51.010  MAIN        ( PROCESS ) : SessionClient(0x7f4edc66e6d0): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:39:51.010  MAIN        ( PROCESS ) : auth_type=None
20.05.20 22:40:02.285  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:56960) response status code: 400
20.05.20 22:40:02.286  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:56960) response message: The query you have entered yields too many matching entries. Please narrow down your search by specifying more query parameters. The current size limit of 2000 can be configured with the UCR variable directory/manager/web/sizelimit.
20.05.20 22:40:02.286  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:56960) response result: None
20.05.20 22:40:02.286  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:56960) response error: {u'traceback': None, u'command': u'Query: udm/query'}
20.05.20 22:40:21.766  MAIN        ( PROCESS ) : SessionClient(0x7f4edc666b10): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:40:21.766  MAIN        ( PROCESS ) : auth_type=None
20.05.20 22:40:30.722  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57040) response status code: 503
20.05.20 22:40:30.722  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57040) response message: The connection to the Univention Management Console Server broke up unexpectedly.
If you have root permissions on the system you can restart UMC by executing the following commands:
 * service univention-management-console-server restart
 * service univention-management-console-web-server restart
Otherwise please contact an administrator or try again later.
20.05.20 22:40:30.722  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57040) response result: None
20.05.20 22:40:38.022  MAIN        ( PROCESS ) : SessionClient(0x7f4edc5a7d90): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:40:38.024  MAIN        ( PROCESS ) : SessionClient(0x7f4edc5a7d90): _authenticated: success=False  status=500  message='NoneType' object has no attribute 'set_item' result=None
20.05.20 22:40:38.025  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57070) response status code: 500
20.05.20 22:40:38.025  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57070) response message: 'NoneType' object has no attribute 'set_item'
20.05.20 22:40:38.025  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57070) response result: None
20.05.20 22:40:38.267  MAIN        ( PROCESS ) : SessionClient(0x7f4edc5a7d90): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:40:38.529  MAIN        ( PROCESS ) : SessionClient(0x7f4edc5a7d90): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:40:38.905  MAIN        ( PROCESS ) : SessionClient(0x7f4edc5a7d90): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:40:39.388  MAIN        ( PROCESS ) : SessionClient(0x7f4edc5a7d90): _authenticated: success=True  status=200  message=None result={u'username': u'Administrator'}
20.05.20 22:40:39.454  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57072) response status code: 401
20.05.20 22:40:39.454  MAIN        ( PROCESS ) : CPCommand (10.205.2.25:57072) response message: Verboten

i dont understand how this can happen
Comment 1 Felix Botner univentionstaff 2020-05-28 11:35:10 CEST
Created attachment 10373 [details]
repr1.sh

simple reproducer
Comment 2 Florian Best univentionstaff 2020-06-09 12:03:30 CEST
The pam instance was potentially shared by multiple threads which could lead to race conditions using the same pam instance. That was bad - it could potentially also cause the wrong language to be used - and has been replaced with a local variable instead of a instance variable. The pam instance is then cleaned up in after the authentication was done.

univention-management-console.yaml
de92e672c7cf | YAML Bug #51366

univention-management-console (11.0.4-82)
befaf1cb0dc0 | Bug #51366: fix leak of open ldap connections in PAM authentication
Comment 3 Felix Botner univentionstaff 2020-06-09 17:36:18 CEST
TODO: yaml
TODO: check jenkins tests

OK - manual tests ok, repr1.sh can no longer reproduce the problem and i also couldn't reproduce the "restart UMC sever after logon" Traceback (comment #1)
Comment 4 Felix Botner univentionstaff 2020-06-11 11:14:55 CEST
OK - yaml
OK - jenkins Tests
Comment 5 Erik Damrose univentionstaff 2020-06-17 15:39:28 CEST
<http://errata.software-univention.de/ucs/4.4/626.html>