Bug 35977 - uvmm libcloud memory consumption
uvmm libcloud memory consumption
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Virtualization - UVMM
UCS 4.0
Other Linux
: P5 normal (vote)
: UCS 4.0
Assigned To: Philipp Hahn
Erik Damrose
: interim-3
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-09-22 15:11 CEST by Erik Damrose
Modified: 2014-11-26 06:54 CET (History)
2 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Erik Damrose univentionstaff 2014-09-22 15:11:27 CEST
For the purpose of instance creation we load all available images via libcloud and cache it in the local memory. Just receiving this information uses 400-500MiB of RAM. The uvmmd node should have 2GiB or more RAM available.

It may be possible to only store a portion of the acquired information, but during the initial request step this much memory will be needed, as paginated or partial transmission of the list is currently not supported.
Comment 1 Erik Damrose univentionstaff 2014-09-22 15:12:38 CEST
Addition: this much ram is needed when requesting ~20000 images
Comment 2 Stefan Gohmann univentionstaff 2014-10-09 18:37:57 CEST
Maybe it is similar to Bug #24273 (r28977).
Comment 3 Philipp Hahn univentionstaff 2014-10-30 14:37:30 CET
libcloud internally uses the ElementTree XML API to parse the large response from AWS. If the most efficient C implementation "lxml.etree" from "pyhton-lxml" is not available, it falls back to the Python internal xml.etree.ElementTree implementation, which seems to be a memory hog, as it requires an explicit call to the garbage collector to get the memory freed:

	scanner.dump_all_objects('1.json')
	d = ec2driver()
	images = d.list_images(ex_owner="")
	scanner.dump_all_objects('2.json')
	gc.collect()
	scanner.dump_all_objects('3.json')
	del images
	scanner.dump_all_objects('4.json')
	gc.collect()
	scanner.dump_all_objects('5.json')
	del d
	scanner.dump_all_objects('6.json')
	gc.collect()
	scanner.dump_all_objects('7.json')

# ./35977_testlibcloud.py ; ls -gGh ?.json
-rw-r--r-- 1 4,4M Okt 30 13:58 1.json
-rw-r--r-- 1 407M Okt 30 13:59 2.json
-rw-r--r-- 1  76M Okt 30 13:59 3.json
-rw-r--r-- 1 4,5M Okt 30 13:59 4.json
-rw-r--r-- 1 4,5M Okt 30 13:59 5.json
-rw-r--r-- 1 4,5M Okt 30 13:59 6.json
-rw-r--r-- 1 4,5M Okt 30 13:59 7.json
# apt-get install python-lxml
# ./35977_testlibcloud.py ; ls -gGh ?.json
-rw-r--r-- 1 4,9M Okt 30 14:26 1.json
-rw-r--r-- 1  76M Okt 30 14:26 2.json
-rw-r--r-- 1  76M Okt 30 14:27 3.json
-rw-r--r-- 1 5,0M Okt 30 14:27 4.json
-rw-r--r-- 1 5,0M Okt 30 14:27 5.json
-rw-r--r-- 1 5,0M Okt 30 14:27 6.json
-rw-r--r-- 1 5,0M Okt 30 14:27 7.json
# lsof -p 10013 | grep lxml
univentio 10013 root  mem    REG              254,0  1209848 22282824 /usr/lib/python2.7/dist-packages/lxml/etree.so
Comment 4 Philipp Hahn univentionstaff 2014-10-30 15:29:25 CET
r55161 | Bug #35977 UVMM: Switch to more efficient python-lxml

Package: univention-virtual-machine-manager-daemon
Version: 4.0.7-1.538.201410301518
Branch: ucs_4.0-0

No changelog needed as libcloud is new in UCS-4.0
Comment 5 Erik Damrose univentionstaff 2014-11-05 10:00:22 CET
OK: python-lxml added to dependencies
OK: no changelog required
OK: reduced memory consumption
Verified
Comment 6 Stefan Gohmann univentionstaff 2014-11-26 06:54:22 CET
UCS 4.0-0 has been released:
 http://docs.univention.de/release-notes-4.0-0-en.html
 http://docs.univention.de/release-notes-4.0-0-de.html

If this error occurs again, please use "Clone This Bug".