Bug 35122 - Automatic refresh of uvmm module
Automatic refresh of uvmm module
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: UMC - Virtual machines (UVMM)
UCS 3.2
Other Linux
: P5 normal (vote)
: UCS 3.2-4-errata
Assigned To: Dirk Wiesenthal
Erik Damrose
:
: 25352 27896 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-06-16 10:28 CEST by Tim Petersen
Modified: 2015-02-18 11:58 CET (History)
5 users (show)

See Also:
What kind of report is it?: ---
What type of bug is this?: ---
Who will be affected by this bug?: ---
How will those affected feel about the bug?: ---
User Pain:
Enterprise Customer affected?:
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number:
Bug group (optional):
Max CVSS v3 score:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Petersen univentionstaff 2014-06-16 10:28:15 CEST
The UVMM Module shouls refresh its server llist automatically.
If its not possible to do this dynamically by events (migrated instance, etc.) it should be updated by configurable interval (UCR).
Comment 1 Alexander Kläser univentionstaff 2014-06-16 13:38:34 CEST
*** Bug 25352 has been marked as a duplicate of this bug. ***
Comment 2 Alexander Kläser univentionstaff 2014-06-16 13:39:18 CEST
*** Bug 27896 has been marked as a duplicate of this bug. ***
Comment 3 Tim Petersen univentionstaff 2014-06-17 15:29:29 CEST
A customer mentions that this refresh has to be done without any negative interaction disturbance (for example while scrolling the list).
Comment 4 Dirk Wiesenthal univentionstaff 2014-07-02 17:15:34 CEST
Fixed in
  univention-virtual-machine-manager-daemon 3.0.17-10.496.201407021649

Changes:
 * Refreshes when page.onShow() <- when going back from details (snapshot, description, ...)
 * Refreshes when clicking on nodes even when UCR says: No autosearch

Migrating, starting, stopping, and everything else should already trigger refresh. Did I miss anything?

No auto update with interval.
Comment 5 Stefan Gohmann univentionstaff 2014-07-02 19:48:39 CEST
(In reply to Dirk Wiesenthal from comment #4)
> No auto update with interval.

Why not? Any particular reason?
Comment 6 Dirk Wiesenthal univentionstaff 2014-07-02 23:02:40 CEST
(In reply to Stefan Gohmann from comment #5)
> (In reply to Dirk Wiesenthal from comment #4)
> > No auto update with interval.
> 
> Why not? Any particular reason?

I regarded it as unnecessary. Real problem were events that could change the state of a machine which did not trigger the reload. This was added.

Now an update interval would only add the benefit to monitor the CPU usage (well, _maybe_ a machine stops running because of a "halt" issued a few seconds ago). And it adds extra complexity. Comment 3 mentions scrolling the list while the autoupdate is triggered. Or just before you wanted to click on a machine (there is a standby animation while reloading).

We do not use this kind of auto update anywhere in UMC. Is it really needed?
Comment 7 Stefan Gohmann univentionstaff 2014-07-03 05:33:38 CEST
(In reply to Dirk Wiesenthal from comment #6)
> > Why not? Any particular reason?
> 
> I regarded it as unnecessary. Real problem were events that could change the
> state of a machine which did not trigger the reload. This was added.

No, not really. When you start a VM, it will have different states and these states are changed without any action at least with Xen. That means if you start a VM one state will be "Paused" and after that the state will be "Running". If the refresh is done when the VM is in state "Pause", it will be shown as "Pause" unless you do other things. That is the customer request.

(In reply to Dirk Wiesenthal from comment #6)
> Now an update interval would only add the benefit to monitor the CPU usage
> (well, _maybe_ a machine stops running because of a "halt" issued a few
> seconds ago). And it adds extra complexity. Comment 3 mentions scrolling the
> list while the autoupdate is triggered. Or just before you wanted to click
> on a machine (there is a standby animation while reloading).

Maybe we could only change the machine state in the refresh?

> We do not use this kind of auto update anywhere in UMC. Is it really needed?

@Philipp, maybe you could talk to the customer?
Comment 8 Philipp Hahn univentionstaff 2014-07-03 09:01:11 CEST
(In reply to Stefan Gohmann from comment #7)
> > We do not use this kind of auto update anywhere in UMC. Is it really needed?
> 
> @Philipp, maybe you could talk to the customer?

The problems were (without verifying it with the customer):

1. When a domain is started, the qemu-dm is launched and connected to the different devices. During that the VM is shown as paused. In that state the VNC button is unavailable (Bug #35107, patch available!), so you can't directly connect to the VM but must first do a manual reload, which is a usability bug.
(UVMMd already waits for 20 seconds for the VM reaching RUNNING, but it still sometimes happens. That time span is IMHO already too long and gives UVMM a very non-responsive feeling).
It's unclear why Xend sometimes takes that long: the problem is probably related to Xend being very inefficient, which is why it got replaced by xen-light with newer versions of Xen.

2. When a VM is migrated between two hosts, the connection between libvirtd on one of the hosts and UVMMd breaks. As a result the host and all its VMs are shown as being offline. This looks broken to the customer, while in fact UVMMd already re-established the connection to libvirtd and is again working fine.
Without a manual reload the host is still shown as offline and its VMs cannot be accessed.

3. UVMM is used for monitoring the VMs, but without a auto-refresh the page is always out-dated. So in addition to switching to the browser and finding the right tab you always have to do a manual refresh before you can start working. Doing a manual refresh once a day might be okay, but having to do it every time you use UVMM feels very annoying.


(In reply to Dirk Wiesenthal from comment #6)
> (well, _maybe_ a machine stops running because of a "halt" issued a few
> seconds ago).

That happens a lot more often then you think. Currently you don't get any notification of state change, so you have to pro-actively do a manual refresh to get the current state.
For a future version something like an event log with time-stamps might help, where you can later see what happened in your environment and were problems appeared. It it is your day-job to keep all VMs running, you probably don't want  to go on a manual bug hunt each X minutes, but get notified of problems.
Comment 9 Erik Damrose univentionstaff 2014-07-03 09:31:34 CEST
Would it be possible to update only changed machine states in the current view? That is: ask uvmmd for the current values, compare with the values currently saved in the grid, and update only the lines which are changed? 

If this avoids redrawing the whole grid then resetting the scrolling state would not be an issue.
Comment 10 Dirk Wiesenthal univentionstaff 2014-07-04 13:05:45 CEST
Fixed in
  univention-virtual-machine-manager-daemon 3.0.17-11.497.201407041252

Grid is updated according to uvmm/umc/autoupdate/interval. Done without refreshing the grid, but by changing the items in this grid.

Every Grid has a store object which holds the items the grid shows. Our implementation of that store is rather limited in that it does not allow new items to be created in Memory.

I would have to mess around with a lot of internals (more than I already did...) to support new virtual machines as well as deleted machines.

In case new machines are created elsewhere or deleted elsewhere, there is just a note to refresh the grid manually.

Everything else (CPU update, state changes, Button updates) should work. Please test different browsers and also check whether it performs acceptable.
Comment 11 Erik Damrose univentionstaff 2014-07-14 17:09:33 CEST
In my test environment, frequent connection problems from uvmmd to the XEN-Nodes are occuring. They should be fixed before releasing this fix. (Bug #35354)

I reopen this bug for the following issues i found:
- The treeview showing the available nodes should be updated with the grid: Otherwise, Nodes may appear offline in the tree, while their VMs are shown as running in the grid view.

- If the uvmmd is not responding, the following traceback is shown every x seconds, where x is the configured autorefresh interval
File "/usr/lib/pymodules/python2.6/notifier/threads.py", line 82, in _run
    tmp = self._function()
  File "/usr/lib/pymodules/python2.6/notifier/__init__.py", line 104, in __call__
    return self._function( *tmp, **self._kwargs )
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/uvmm/uvmmd.py", line 84, in request
    raise UVMM_Error(str(ex))
UVMM_Error: Could not open socket "/var/run/uvmm.socket": 2

The traceback should be caught, and only one popup should be shown.

- Firefox performance: Firefox uses about 85% CPU on my machine while displaying the hosts in my test environment, while chromium uses about 10%. Is there room for performance improvements?

- If a host migration takes a certain amount of time, the message "The number of virtual machines changed..." appears. This is misleading, as all VMs are correctly shown after the migration is complete. This may be an issue in uvmmd/libvirt while migrating the VM; It is concurrently defined at the source and target VM-Node. I do not know how easy it would be to hide this in the frontend.


The automatic list refreshing itself works great, there are no scrolling issues, as requested.
Comment 12 Erik Damrose univentionstaff 2014-07-15 14:34:39 CEST
Please ask me or Stefan before working on this bug again.
Comment 13 Philipp Hahn univentionstaff 2014-07-15 15:23:51 CEST
(In reply to Erik Damrose from comment #12)
> Please ask me or Stefan before working on this bug again.
<http://bladis/gdkZ7xVi8Y>
Comment 14 Erik Damrose univentionstaff 2014-07-16 13:10:45 CEST
One additional note: Is it possible that due to the autorefresh the user will not get logged out after the umc session timeout (defined in umc/http/session/timeout)? It seems like the autorefresh is seen as a user action.
Comment 15 Dirk Wiesenthal univentionstaff 2014-07-16 13:36:25 CEST
Firefox Performance: can you confirm that these 85% only happen with autoupdate? FF has issues with our grid in general.

Logout: have not checked but IIRC there is little I can do. Each request, whether user induced or started "in the background", resets the session. Holds for frontend (JS) and backend (umc-server).
Comment 16 Erik Damrose univentionstaff 2014-10-08 13:20:56 CEST
For UCS4:
r54250 estroy GridUpdater when module gets closed
Comment 17 Florian Best univentionstaff 2014-10-08 13:21:10 CEST
I fixed the removal of the GridUpdater (in UCS4.0) when the module gets closed. Otherwise the requests run forever. (The error handling is btw. currently completely missing.)
svn54250
* Bug #35122: destroy GridUpdater when module gets closed

dwiesent: FYI: this.own() works only when the object inherits from dijit/Destroyable → don't just put this.own() around everything (e.g. in umc.widgets.Grid was the same failure).
Comment 18 Stefan Gohmann univentionstaff 2014-10-28 22:18:47 CET
Moved to UCS 3.2-4-errata. It is already part of UCS 4, but I think it is no problem.
Comment 19 Dirk Wiesenthal univentionstaff 2015-02-10 13:13:27 CET
Re-added the code that was reverted and also merged a fix from fbest. Should work as in UCS 4.0-0 now.
  univention-virtual-machine-manager-daemon 3.0.19-2.582.201502101307
Comment 20 Erik Damrose univentionstaff 2015-02-17 09:53:17 CET
Works very well in general. But error handling should be improved -> reopen. This was already fixed in Bug 33963 for UCS 4 and be backported. Have a look at the bug for svn revisions with the fixes.
Comment 21 Dirk Wiesenthal univentionstaff 2015-02-17 10:16:05 CET
Backported in
  univention-virtual-machine-manager-daemon 3.0.19-3.585.201502170959
Comment 22 Erik Damrose univentionstaff 2015-02-17 11:08:22 CET
It seems only the UMC frontend part was backported. There are still repeated Could not open socket "/var/run/uvmm.socket" popups.
Comment 23 Dirk Wiesenthal univentionstaff 2015-02-17 13:53:58 CET
Completely backported Bug#33963 in
  univention-virtual-machine-manager-daemon 3.0.19-4.586.201502171348
Comment 24 Erik Damrose univentionstaff 2015-02-17 14:54:37 CET
If i open the UVMM module there is an immediate traceback:

Konnte das Modul uvmm nicht laden: cannot import name error_handling
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/univention/management/console/protocol/modserver.py", line 95, in _load_module
    self.__module = __import__(file_, [], [], modname)
  File "/usr/lib/pymodules/python2.6/univention/management/console/modules/uvmm/__init__.py", line 34, in <module>
    from univention.management.console.modules import Base, UMC_OptionTypeError, error_handling
ImportError: cannot import name error_handling
Comment 25 Florian Best univentionstaff 2015-02-17 15:01:20 CET
You can't backport the changes because I added such things during UCS 4.0 development to the UMC-server core.
Comment 26 Dirk Wiesenthal univentionstaff 2015-02-17 16:41:28 CET
Fixed in
  univention-virtual-machine-manager-daemon 3.0.19-6.588.201502171636
Comment 27 Erik Damrose univentionstaff 2015-02-17 17:10:37 CET
OK: Auto refresh works as in UCS 4
OK: Yaml
Verified
Comment 28 Moritz Muehlenhoff univentionstaff 2015-02-18 11:58:34 CET
http://errata.univention.de/ucs/3.2/287.html