Bug 44965 - UMC-Webserver crashes with "[Errno24] Too many open files" at login
UMC-Webserver crashes with "[Errno24] Too many open files" at login
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC (Generic)
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.2-1-errata
Assigned To: Florian Best
Alexander Kläser
:
Depends on:
Blocks: 45307
  Show dependency treegraph
 
Reported: 2017-07-11 12:39 CEST by Richard Ulmer
Modified: 2017-08-31 13:31 CEST (History)
5 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 2: Will only affect a few installed domains
How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.400
Enterprise Customer affected?: Yes
School Customer affected?: Yes
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number: 2017071121000575, 2017080721000143, 2017081421000335
Bug group (optional): Security, Troubleshooting, Usability
Max CVSS v3 score:
best: Patch_Available+


Attachments
Screenshot of the traceback, that appeared when trying to log in (58.17 KB, image/png)
2017-07-11 12:39 CEST, Richard Ulmer
Details
result of the lsof command (120.62 KB, text/plain)
2017-07-11 12:40 CEST, Richard Ulmer
Details
patch (891 bytes, patch)
2017-08-14 13:43 CEST, Florian Best
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Ulmer univentionstaff 2017-07-11 12:39:29 CEST
Created attachment 9010 [details]
Screenshot of the traceback, that appeared when trying to log in

After opening many umc-modules I was not able to login to the umc anymore. I got a traceback (see screenshot.png).

Restarting the webserver solved the problem.
Comment 1 Richard Ulmer univentionstaff 2017-07-11 12:40:19 CEST
Created attachment 9011 [details]
result of the lsof command
Comment 2 Richard Ulmer univentionstaff 2017-07-11 12:42:21 CEST
Executing `lsof -p $PID_WEBSERVER`, where $PID_WEBSERVER comes from `pgrep -fl web-server` gave the output that can be seen in the attachment result_of_the_lsof_command.
Comment 3 Nico Stöckigt univentionstaff 2017-08-07 10:27:07 CEST
Happend also in a Customer Environment with UCS 4.2-1 e118
Comment 4 Florian Best univentionstaff 2017-08-07 11:09:29 CEST
(In reply to Nico Stöckigt from comment #3)
> Happend also in a Customer Environment with UCS 4.2-1 e118
If possible please attach a lsof output of the process.
Comment 5 Sönke Schwardt-Krummrich univentionstaff 2017-08-14 12:34:14 CEST
Added traceback to comment for easier bug search:

Traceback (most recent call last):
  File "/usr/sbin/univention-management-console-web-server", line 235, in check_queue
    cls.dispatch(queuerequest)
  File "/usr/sbin/univention-management-console-web-server", line 250, in dispatch
    client = SessionClient(ip=queuerequest.ip)
  File "/usr/sbin/univention-management-console-web-server", line 126, in __init__
    self.client = Client()
  File "/usr/lib/pymodules/python2.7/univention/management/console/protocol/client.py", line 119, in __init__
  File "/usr/lib/pymodules/python2.7/univention/management/console/protocol/client.py", line 143, in _init_socket
  File "/usr/lib/python2.7/socket.py", line 187, in __init__
    _sock = _realsocket(family, type, proto)
error: [Errno 24] Too many open files

Occured several times at UCS@school customer

"lsof -i $PID-UMC-WEBSERVER ; lsof -i $PID-UMC-SERVER ; ps faxwww" is available at ticket 2017081421000335
Comment 7 Florian Best univentionstaff 2017-08-14 13:43:22 CEST
Created attachment 9116 [details]
patch
Comment 8 Florian Best univentionstaff 2017-08-14 13:46:31 CEST
univention-management-console.yaml:
r82101 | YAML Bug #44965

univention-management-console (9.0.80-62):
r82100 | Bug #44965: fix a memory leak in the UMC-Server process
Comment 9 Florian Best univentionstaff 2017-08-14 13:55:35 CEST
The problem lies in the UMC-Server, which didn't close the connections after 30 seconds if a request was send whichs response could be send in one chunk to the client. The response wasn't closed because state.requests still contained the response which was already answered but the logic in _timeout_connection() expects an empty state.requests to close the connection.
Comment 10 Alexander Kläser univentionstaff 2017-08-16 11:07:56 CEST
Works. YAML file is OK.

→ VERIFIED
Comment 11 Arvid Requate univentionstaff 2017-08-16 11:49:43 CEST
check_errata_for_release sagt:

[FAIL] bug.43658.scope+version: Bug 43658 scope: ---
[FAIL] changes.valid: Mismatching binary package version: 9.0.80-63A~4.2.0.2017081413 != univention-management-console 9.0.80-63A~4.2.0.201708141353 from univention-management-console 9.0.80-63A~4.2.0.201708141353

Im Scope ist 9.0.80-63A~4.2.0.201708141353 gebaut
Comment 12 Arvid Requate univentionstaff 2017-08-16 11:50:30 CEST
Vielleicht war das auch Bug 43658, egal, bitte einfach prüfen.
Comment 13 Florian Best univentionstaff 2017-08-16 11:57:28 CEST
r82167
Comment 14 Erik Damrose univentionstaff 2017-08-16 12:44:25 CEST
<http://errata.software-univention.de/ucs/4.2/139.html>