Bug 45309 - Too many open files in UMC
Too many open files in UMC
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.1
Other Linux
: P5 normal (vote)
: UCS 4.2-2-errata
Assigned To: Florian Best
Eduard Mai
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-09-01 07:26 CEST by Stefan Gohmann
Modified: 2017-10-10 15:38 CEST (History)
1 user (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 3: A User would likely not purchase the product
User Pain: 0.240
Enterprise Customer affected?:
School Customer affected?: Yes
ISV affected?:
Ticket number: 2017071121000575
Bug group (optional):
Max CVSS v3 score:


Attachments
check-max-open-files.sh (463 bytes, text/plain)
2017-09-01 07:32 CEST, Stefan Gohmann
Details
umc_soft_limits.patch (554 bytes, patch)
2017-09-01 07:33 CEST, Stefan Gohmann
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Gohmann univentionstaff 2017-09-01 07:26:49 CEST
We received the following UMC abort from a customer environment:

08.07.17 15:36:27.299  MAIN        ( ERROR   ) : Error during handling a
request: Traceback (most recent call last):
  File
"/usr/lib/pymodules/python2.7/univention/management/console/protocol/server.py",
line 155, in _receive
    self._handle(state, msg)
  File
"/usr/lib/pymodules/python2.7/univention/management/console/protocol/server.py",
line 204, in _handle
    Server.reload()
  File
"/usr/lib/pymodules/python2.7/univention/management/console/protocol/server.py",
line 457, in reload
    moduleManager.load()
  File
"/usr/lib/pymodules/python2.7/univention/management/console/module.py",
line 391, in load
OSError: [Errno 24] Too many open files:
'/usr/share/univention-management-console/modules
Comment 1 Stefan Gohmann univentionstaff 2017-09-01 07:31:05 CEST
After some more debugging, we found that the soft limit for open file descriptors was reached:

PID [PROCESS]	 limit	Hard limit	Open files
[...]
689 [/usr/bin/python2.7/usr/sbin/univention-management-console-serverrestart]	4096	64512	4297

The soft limit is set directly in UMC so that the limits settings are ignored (/etc/security/limits.conf).
Comment 2 Stefan Gohmann univentionstaff 2017-09-01 07:32:55 CEST
Created attachment 9161 [details]
check-max-open-files.sh

Simple script to check the max open file settings.
Comment 3 Stefan Gohmann univentionstaff 2017-09-01 07:33:43 CEST
Created attachment 9162 [details]
umc_soft_limits.patch

Patch to increase the soft limit:
 patch -p0 -d / <umc_soft_limits.patch
 /etc/init.d/univention-management-console-server restart
Comment 4 Florian Best univentionstaff 2017-09-20 17:26:15 CEST
We introduced this code in Bug #39909. UMC never took any limits from /etc/security/limits.conf.
The default value of UCS is:
*                 soft     nofile   32768
*                 hard     nofile   32768

but somehow these limits aren't evaluated by python(?) processes.
I get the following limits if I remove the code:
Max open files            1024                 4096                 files

So I will take Stefans patch.
Comment 5 Florian Best univentionstaff 2017-09-20 17:34:26 CEST
univention-management-console (9.0.80-71):
836f0ab374b7 | Merge branch 'fbest/45309-umc-soft-limits' into 4.2-2
399cddd08e52 | Bug #45309: raise soft limit for max open files of the UMC server

univention-management-console.yaml:
baa9d275dc8a | YAML Bug #45309
Comment 6 Eduard Mai univentionstaff 2017-09-22 14:39:16 CEST
OK, works. UMC Server is able to hold the desired amount of file descriptors as of 9.0.80-71:

27299 [/usr/bin/python2.7/usr/sbin/univention-management-console-serverstart]	64512	64512	64690

VERIFIED
Comment 7 Arvid Requate univentionstaff 2017-10-10 15:38:47 CEST
<http://errata.software-univention.de/ucs/4.2/196.html>