Univention Bugzilla – Bug 52371
Add multiprocessing to UMC-Server
Last modified: 2020-12-16 16:03:33 CET
The UMC-Server should also be adjusted to start multiple processes. +++ This bug was initially created as a clone of Bug #52293 +++ The UMC (web) server is a Python process with a single-thread main loop, so not much more than one CPU core can be used by this process. To scale better in very large environments with many requests, the UMC processes should be started multiple times to be able to use multiple CPU cores.
The UMC-Webserver now forks multiple UMC processes by setting the UCR variable umc/server/processes. The signals for reloading and log rotate are forwarded to all child processes. When multiprocessing is enabled the PID is also logged by UMC's univention.debug implementation. We saw one hanging extra process in a accept() call due to the multiprocessing.Manager(). According to Sönke this is okay for now. univention-management-console (11.0.5-24) 90ae5a5f7bb6 | Bug #52371: make UMC-Server multiprocessing capable 0342ef5a4ff6 | Bug #52371: add possibility to log also the process PID univention-management-console.yaml 548722752eab | YAML Bug #52371, Bug #52443, Bug #52442, Bug #52444, Bug #52293, Bug #52272
Created attachment 10575 [details] Possible patch Looks good I noticed two small problems. But they are corner cases that shouldn't happen in production, so I'm not sure if they need to be fixed. 1) If a umc worker process gets terminated manually, the other process won't be terminated but the server.socket gets removed and no new connections are possible, which is a bit confusing. 2) If a umc worker dies or is killed, the parent process terminates without terminating the other children. Again no new connections are possible. Do these need to be fixed?
Thank you, I took your patch. univention-management-console (11.0.5-26) a94deb0bad3c | Bug #52371: debian/changelog af0e961739c1 | Bug #52371: die when child process crashes ecd2eb2835ea | Bug #52371: remove UNIX socket only in parent process
Sorry for the error in the patch. Tests are looking good now :) What I tested: Upgrade -> OK Login -> OK Login SAML -> OK users/user -> OK portal login -> OK portal edit -> OK jenkins -> OK Setting "umc/server/processes=5" -> 7 umc processes -> OK (See note) Using SIGSTOP and SIGCONT to check that all server process requests -> OK Apache load balancer cookie -> OK systemctl restart univention-management-console-server.service -> OK Merge 5.0 -> OK YAML -> OK Note: This adds two additional processes: - One control process from tornado which forks the umc-servers - One server process from the python multiprocessing Manager, for shared memory
A version bump for u-apache is required to ensure the package update will be done in all scenarios 8a184aaf version bump 0347c929 yaml univention-apache 11.0.2-1A~4.4.0.202012152249
(In reply to Erik Damrose from comment #5) > A version bump for u-apache is required to ensure the package update will be > done in all scenarios > > 8a184aaf version bump > 0347c929 yaml > univention-apache 11.0.2-1A~4.4.0.202012152249 Thanks! I also rebuild univention-web (3.0.6-1A~4.4.0.202012152327) and univention-saml (6.0.3-1A~4.4.0.202012152332) What I tested: Only simplesamlphp (no changes?) and univention-python (bug #52298 still open) still have a higher version in the customers scope -> OK
<https://errata.software-univention.de/#/?erratum=4.4x844> <https://errata.software-univention.de/#/?erratum=4.4x845> <https://errata.software-univention.de/#/?erratum=4.4x847> <https://errata.software-univention.de/#/?erratum=4.4x848>