Bug 30659 - Free memory calculation does not always work
Free memory calculation does not always work
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - App-Center
UCS 3.1
Other Linux
: P3 normal (vote)
: UCS 3.1-1-errata
Assigned To: Dirk Wiesenthal
Felix Botner
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-03-05 12:05 CET by Tim Petersen
Modified: 2013-05-22 16:10 CEST (History)
4 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Flags outvoted (downgraded) after PO Review:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Petersen univentionstaff 2013-03-05 12:05:59 CET
As described at Ticket #2013030121000072 the free memory calculation at the app center does not work on all systems:

Traceback (most recent call last):
   File "/usr/lib/pymodules/python2.6/univention/management/console/modules/__init__.py",
line 204, in execute
     func( request )
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py", line
176, in _response
     return function(self, request)
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py", line
282, in _response
     result = _multi_response(self, request)
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py", line
384, in _response
     for res in function(self, iterator, *nones):
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/decorators.py", line
266, in _fake_func
     yield function(self, *args)
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/__init__.py",
line 130, in query
     props = application.to_dict(self.package_manager)
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/app_center.py",
line 467, in to_dict
     res['cannot_install_reason'], res['cannot_install_reason_detail'] =
self.cannot_install_reason(package_manager)
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/app_center.py",
line 519, in cannot_install_reason
     elif self.get('minphysicalram') and get_current_ram_available() <
self.get('minphysicalram'):
   File
"/usr/lib/pymodules/python2.6/univention/management/console/modules/appcenter/util.py",
line 166, in get_current_ram_available
     phymem_buffers = meminfo['Buffers:']
KeyError: 'Buffers:'


The customer uses openvz - it seems that there is no "Buffers" line in /proc/meminfo when using openvz. Perhaps this is also the case on other virtualization solutions.
Comment 1 Dirk Wiesenthal univentionstaff 2013-03-15 09:30:13 CET
In openvz the App Center is completely unusable. The Test will raise even when getting all applications in the overview.

The question is if we can get the correct number from somewhere.

Of course meminfo.get('Buffers:', 0) will solve the problem, but it may very well be that in this environment the Apps are not installable.
Comment 2 Alexander Kläser univentionstaff 2013-03-15 10:00:33 CET
(In reply to comment #1)
> In openvz the App Center is completely unusable. The Test will raise even when
> getting all applications in the overview.
> 
> The question is if we can get the correct number from somewhere.
> 
> Of course meminfo.get('Buffers:', 0) will solve the problem, but it may very
> well be that in this environment the Apps are not installable.

What about the python module psutils? It seems that the available memory can be easily queried there (property available):

http://code.google.com/p/psutil/wiki/Documentation#Memory
Comment 3 Dirk Wiesenthal univentionstaff 2013-03-15 10:14:48 CET
(In reply to comment #2)
> (In reply to comment #1)
> > In openvz the App Center is completely unusable. The Test will raise even when
> > getting all applications in the overview.
> > 
> > The question is if we can get the correct number from somewhere.
> > 
> > Of course meminfo.get('Buffers:', 0) will solve the problem, but it may very
> > well be that in this environment the Apps are not installable.
> 
> What about the python module psutils? It seems that the available memory can be
> easily queried there (property available):
> 
> http://code.google.com/p/psutil/wiki/Documentation#Memory

Our psutil is outdated. It does not provide the required function. And even if it did: I used the same routines to get the available memory as psutil does in a newer version.
Comment 4 Alexander Kläser univentionstaff 2013-03-15 11:23:05 CET
(In reply to comment #3)
> Our psutil is outdated. It does not provide the required function. And even if
> it did: I used the same routines to get the available memory as psutil does in
> a newer version.

But then we could open a bug for psutils ;) .
Comment 5 Dirk Wiesenthal univentionstaff 2013-04-04 18:12:41 CEST
I did not manage to get OpenVZ kernel running (documentation quite RHEL centric). But I think Buffers are just 0 in OpenVZ and the calculation is still correct:

http://openvz.livejournal.com/35628.html

Buffers are missing in the output, but MemFree + Cached = certain $(free -m)
Comment 6 Dirk Wiesenthal univentionstaff 2013-04-10 20:18:16 CEST
Built for 3.1-1:
  univention-management-console-module-appcenter 2.0.139-2.120.201304102007

Changelog updated, YAML created
Comment 7 Felix Botner univentionstaff 2013-05-15 17:47:35 CEST
http://www.vtk.org/Bug/view.php?id=13227 describes that even "Cached:"  is not always available in /proc/meminfo in a openvz container

/proc/meminfo
MemTotal: 12000000 kB
MemFree: 11852352 kB
SwapTotal: 0 kB
SwapFree: 0 kB

Why not meminfo.get('Cached:', 0) instead meminfo['Cached:'] (as for Buffers:)?
Comment 8 Dirk Wiesenthal univentionstaff 2013-05-17 17:44:02 CEST
Yes, see also http://code.google.com/p/psutil/issues/detail?id=313

I am not sure whether the calculation is still correct under these circumstances but it is probably better than raising.

Fixed in
  univention-management-console-module-appcenter 2.0.151-1.140.201305171739
and
  univention-management-console-module-appcenter 2.0.139-13.139.201305171736
Comment 9 Felix Botner univentionstaff 2013-05-21 09:50:04 CEST
OK - errata3.1-1
OK - yaml

OK - ucs3.1-2
OK - changelog
Comment 10 Moritz Muehlenhoff univentionstaff 2013-05-22 16:10:00 CEST
http://errata.univention.de/ucs/3.1/102.html