Bug 45309

Summary: Too many open files in UMC
Product: UCS Reporter: Stefan Gohmann <gohmann>
Component: UMC (Generic)Assignee: Florian Best <best>
Status: CLOSED FIXED QA Contact: Eduard Mai <mai>
Severity: normal    
Priority: P5 CC: best
Version: UCS 4.1   
Target Milestone: UCS 4.2-2-errata   
Hardware: Other   
OS: Linux   
What kind of report is it?: Bug Report What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains How will those affected feel about the bug?: 3: A User would likely not purchase the product
User Pain: 0.240 Enterprise Customer affected?:
School Customer affected?: Yes ISV affected?:
Waiting Support: Flags outvoted (downgraded) after PO Review:
Ticket number: 2017071121000575 Bug group (optional):
Max CVSS v3 score:
Attachments: check-max-open-files.sh
umc_soft_limits.patch

Description Stefan Gohmann univentionstaff 2017-09-01 07:26:49 CEST
We received the following UMC abort from a customer environment:

08.07.17 15:36:27.299  MAIN        ( ERROR   ) : Error during handling a
request: Traceback (most recent call last):
  File
"/usr/lib/pymodules/python2.7/univention/management/console/protocol/server.py",
line 155, in _receive
    self._handle(state, msg)
  File
"/usr/lib/pymodules/python2.7/univention/management/console/protocol/server.py",
line 204, in _handle
    Server.reload()
  File
"/usr/lib/pymodules/python2.7/univention/management/console/protocol/server.py",
line 457, in reload
    moduleManager.load()
  File
"/usr/lib/pymodules/python2.7/univention/management/console/module.py",
line 391, in load
OSError: [Errno 24] Too many open files:
'/usr/share/univention-management-console/modules
Comment 1 Stefan Gohmann univentionstaff 2017-09-01 07:31:05 CEST
After some more debugging, we found that the soft limit for open file descriptors was reached:

PID [PROCESS]	 limit	Hard limit	Open files
[...]
689 [/usr/bin/python2.7/usr/sbin/univention-management-console-serverrestart]	4096	64512	4297

The soft limit is set directly in UMC so that the limits settings are ignored (/etc/security/limits.conf).
Comment 2 Stefan Gohmann univentionstaff 2017-09-01 07:32:55 CEST
Created attachment 9161 [details]
check-max-open-files.sh

Simple script to check the max open file settings.
Comment 3 Stefan Gohmann univentionstaff 2017-09-01 07:33:43 CEST
Created attachment 9162 [details]
umc_soft_limits.patch

Patch to increase the soft limit:
 patch -p0 -d / <umc_soft_limits.patch
 /etc/init.d/univention-management-console-server restart
Comment 4 Florian Best univentionstaff 2017-09-20 17:26:15 CEST
We introduced this code in Bug #39909. UMC never took any limits from /etc/security/limits.conf.
The default value of UCS is:
*                 soft     nofile   32768
*                 hard     nofile   32768

but somehow these limits aren't evaluated by python(?) processes.
I get the following limits if I remove the code:
Max open files            1024                 4096                 files

So I will take Stefans patch.
Comment 5 Florian Best univentionstaff 2017-09-20 17:34:26 CEST
univention-management-console (9.0.80-71):
836f0ab374b7 | Merge branch 'fbest/45309-umc-soft-limits' into 4.2-2
399cddd08e52 | Bug #45309: raise soft limit for max open files of the UMC server

univention-management-console.yaml:
baa9d275dc8a | YAML Bug #45309
Comment 6 Eduard Mai univentionstaff 2017-09-22 14:39:16 CEST
OK, works. UMC Server is able to hold the desired amount of file descriptors as of 9.0.80-71:

27299 [/usr/bin/python2.7/usr/sbin/univention-management-console-serverstart]	64512	64512	64690

VERIFIED
Comment 7 Arvid Requate univentionstaff 2017-10-10 15:38:47 CEST
<http://errata.software-univention.de/ucs/4.2/196.html>