Bug 52361 - Memory Leak in univention-management-console-server: LDAP connections are not cleaned up correctly
Memory Leak in univention-management-console-server: LDAP connections are not...
Status: RESOLVED DUPLICATE of bug 43633
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.4
Other Linux
: P5 normal (vote)
: ---
Assigned To: Florian Best
UMC maintainers
https://git.knut.univention.de/univen...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2020-11-13 12:32 CET by Florian Best
Modified: 2024-03-07 09:03 CET (History)
0 users

See Also:
What kind of report is it?: Development Internal
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional): Large environments, Security, UCS Performance
Max CVSS v3 score:


Attachments
WIP patch (3.55 KB, patch)
2020-11-13 12:42 CET, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Best univentionstaff 2020-11-13 12:32:55 CET
We are still experiencing some memory leaks in the UMC-Server.

Possible candidate/cause:

umcp.session.ProcessorBase._reload_acls() has a ldap connection instance, which is referencing the LDAP_ACL's class. The _reload_acls() throws away the old LDAP_ACL's instance, which then is never cleaned up.

This might be triggered by "get/modules" via
self._reload_acls_and_permitted_commands.
Comment 1 Florian Best univentionstaff 2020-11-13 12:42:43 CET
Created attachment 10555 [details]
WIP patch

During working on Bug #50583 I already created a fix:
https://git.knut.univention.de/univention/ucs/-/commit/0050fc80d49ae43d1e82b38a24ecc22010e7a4ee

I considered this uncomplete, maybe broken, so I did not take this.
Comment 3 Florian Best univentionstaff 2022-05-06 10:52:10 CEST
On one of our internal servers (UCS: 4.4-9 errata1233) has symptoms:

# lsof -p 5707 | grep laiva.knut.univention.de:7389 | wc -l
11591
# lsof -p 5707 | grep laiva.knut.univention.de:7389 | grep CLOSE_WAIT |  wc -l
5722
# lsof -p 5707 | grep laiva.knut.univention.de:7389 | grep ESTABLISHED |  wc -l
6016

The logs are full of messages like:
06.05.22 10:27:12.116  MAIN        ( WARN    ) : Module None (command=u'uvmm/query', id=u'165182563204742-873172') does not exists anymore
06.05.22 10:27:12.130  MAIN        ( WARN    ) : Module None (command=u'uvmm/node/query', id=u'165182563204952-873173') does not exists anymore
06.05.22 10:27:12.139  MAIN        ( WARN    ) : Module None (command=u'uvmm/cloud/query', id=u'165182563205075-873174') does not exists anymore

# pyrasite-shell 5707
>>> len(umc_daemon.server._Server__bucket._MagicBucket__states)
92
>>> len([s for s in umc_daemon.server._Server__bucket._MagicBucket__states.values() if s.session.processor is not None])                                                                                           
14
>>> set([s.session.processor.acls.lo.lo.lo.whoami_s() for s in umc_daemon.server._Server__bucket._MagicBucket__states.values() if s.session.processor is not None])
set(['dn:cn=****,cn=dc,cn=computers,dc=base'])
>>> import univention
>>> univention.management.console.ldap.user_connection.im_self._LDAP__ldap_connections
{}
→ this must be the issue mentioned in comment #0.
Comment 4 Florian Best univentionstaff 2022-05-06 16:03:58 CEST
(In reply to Florian Best from comment #3)
> The logs are full of messages like:
> 06.05.22 10:27:12.116  MAIN        ( WARN    ) : Module None (command=u'uvmm/query', id=u'165182563204742-873172') does not exists anymore
> 06.05.22 10:27:12.130  MAIN        ( WARN    ) : Module None (command=u'uvmm/node/query', id=u'165182563204952-873173') does not exists anymore
> 06.05.22 10:27:12.139  MAIN        ( WARN    ) : Module None (command=u'uvmm/cloud/query', id=u'165182563205075-873174') does not exists anymore
My theory about this:
The UVMM module doesn't stop the polling if one is logged out (e.g. due to SAML session end time (=5 minutes), UMC-Webserver restarts, etc.)
This causes that the request is made but the reloaded ACL's don't know this module anymore and therefore this message is raised.
The code then does:
 493 »   »   »   CORE.warn('Module %r (command=%r, id=%r) does not exists anymore' % (module_name, command, msg.id))
 494 »   »   »   moduleManager.load()
 495 »   »   »   self._reload_acls_and_permitted_commands()

which was causing the memory leak as explained in comment 0.
Comment 6 Florian Best univentionstaff 2023-06-19 14:16:50 CEST
Patch was applied in Bug #43633.

*** This bug has been marked as a duplicate of bug 43633 ***