Univention Bugzilla – Full Text Bug Listing |
Summary: | Memory leak in UVMM daemon | ||
---|---|---|---|
Product: | UCS | Reporter: | Alexander Kläser <klaeser> |
Component: | Virtualization - UVMM | Assignee: | Philipp Hahn <hahn> |
Status: | CLOSED FIXED | QA Contact: | Erik Damrose <damrose> |
Severity: | normal | ||
Priority: | P5 | CC: | best, damrose, fricke, gohmann, peichert, walkenhorst |
Version: | UCS 4.0 | ||
Target Milestone: | UCS 4.0-1-errata | ||
Hardware: | Other | ||
OS: | Linux | ||
See Also: | https://forge.univention.org/bugzilla/show_bug.cgi?id=35977 | ||
What kind of report is it?: | --- | What type of bug is this?: | --- |
Who will be affected by this bug?: | --- | How will those affected feel about the bug?: | --- |
User Pain: | Enterprise Customer affected?: | ||
School Customer affected?: | ISV affected?: | ||
Waiting Support: | Flags outvoted (downgraded) after PO Review: | ||
Ticket number: | Bug group (optional): | Troubleshooting | |
Max CVSS v3 score: | |||
Attachments: |
Round-Robin Database: UVMMd Virtual Memory usage
Simple test script |
Description
Alexander Kläser
2014-11-13 13:27:05 CET
Do you have any logfiles to analyse this behaviour? UVMMd does require a large amount of memory, especially if EC2 connections are configured. But we did not see a memory-leak in uvmm-daemon itself. See Bug #35977 comment 3 for a UVMMd memory leak with form info on debugging that kind of problems using "python-meliae". Reported via feedback: Ticket #2015021821000235 More feedback from the partner: They have three KVM servers which are managed through UVMMd. UVMMd runs on two systems. Using meliae I've found that xml.dom.minidom seems to have a memory leak: <http://stackoverflow.com/questions/26787026/memory-leak-parsing-xml-using-xml-dom-minidom> r59486 | Bug #36640 UVMM: Fix ucslint errors/warning r59485 | Bug #36640 UVMM: Bump version r59483 | Bug #36640 UVMM: Fix memory leak in XML parsing r59482 | Bug #36640 UVMM: Split/add unit test r59481 | Bug #36640 UVMM: Add memory debugger Package: univention-virtual-machine-manager-daemon Version: 4.0.24-2.592.201503291138 Branch: ucs_4.0-0 Scope: errata4.0-1 r59487 | Bug #36635,Bug #36640 UVMM: YAML 2015-03-29-univention-virtual-machine-manager-daemon.yaml r59498 | Bug #36640 UVMM: Fix memory leak in XML parsing lxml does not allow setting an attribute to None Package: univention-virtual-machine-manager-daemon Version: 4.0.24-3.593.201503301426 Branch: ucs_4.0-0 Scope: errata4.0-1 r59499 | Bug #36640 UVMM: Fix memory leak in XML parsing YAML 2015-03-29-univention-virtual-machine-manager-daemon.yaml OK: r59498 | Bug #36640 UVMM: Fix memory leak in XML parsing lxml does not allow setting an attribute to None -> Creating new instances is once more possible Reopen: As discussed, the memory leak is still present, as tested with the most recent version on laiva # while cat /proc/$(</etc/runit/univention-virtual-machine-manager-daemon/supervise/pid)/statm;do sleep 1;done shows VmRSS to increase permanently in fixed intervals. See <http://xen12.knut.univention.de/collectd/bin/index.cgi?hostname=laiva.knut.univention.de&plugin=processes×pan=86400&action=show_selection&ok_button=OK> for a long-term trend. # gdb -p ... -ex 'generate-core-file' ; strings did show a lot of XML data. # valgrind --leak-check=full --show-reachable=yes --log-file=val.log val.py shows XMLDesc() leaking memory. r14565 | Bug #36640 libvirt: Memory leak Upstream commit 4acfb169400497da2a82a849dc8aaa65f88ac6d1 fixed at least one huge memory leak, but the memory usage of UVMMd is still not stable (= increasing). I will let collectd run some more to get a longer trend as I'm not yet convinced that the problem is fixed now. $ repo_admin.py --cherrypick -r 4.0 --releasedest 4.0 --dest errata4.0-1 -p libvirt-python Package: libvirt-python Version: 1.2.6-2.3.201504081137 Branch: ucs_4.0-0 Scope: errata4.0-1 r59644 | Bug #36640 libvirt-python: Memory leak 2015-04-08-libvirt-python.yaml r59631 | Bug #36640 UVMM: Fix optional keymap in XML parsing Package: univention-virtual-machine-manager-daemon Version: 4.0.24-7.597.201504081138 Branch: ucs_4.0-0 Scope: errata4.0-1 Created attachment 6812 [details] Round-Robin Database: UVMMd Virtual Memory usage UVMMd still shows a memory leak, but a lot less then before. # rrdtool graph ps_vm.png -s '09:00 2015-04-07' -e '09:00 2015-04-09' DEF:value=ps_vm.rrd:value:AVERAGE LINE1:value#0000FF:"PsVm" # display ps_vm.png If it becomes a problem again, clone this bug for further memory leak debugging: * Use Valgrind * Suppress "use-once" allocations <http://valgrind.org/docs/manual/manual-core.html#manual-core.suppress> Created attachment 6813 [details]
Simple test script
The program shows an ever increasing memory usage.
(In reply to Philipp Hahn from comment #10) > Created attachment 6813 [details] > Simple test script > > The program shows an ever increasing memory usage. That's wrong: python2.7-dbg has the memory leak - the script is fine, so no easy way to re-produce the leak so far. OK: YAML r59644 | Bug #36640 libvirt-python: Memory leak OK: r59631 | Bug #36640 UVMM: Fix optional keymap in XML parsing OK: r14565 | Bug #36640 libvirt: Memory leak -> Verified |