Bug 22695 - VM gets paused on disk full ENOSPC
VM gets paused on disk full ENOSPC
Status: CLOSED FIXED
Product: UCS
Classification: Unclassified
Component: Virtualization - UVMM
UCS 4.2
Other Linux
: P5 normal (vote)
: UCS 4.3-2-errata
Assigned To: Philipp Hahn
Erik Damrose
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-06-09 11:59 CEST by Moritz Muehlenhoff
Modified: 2018-12-05 14:38 CET (History)
4 users (show)

See Also:
What kind of report is it?: Bug Report
What type of bug is this?: 7: Crash: Bug causes crash or data loss
Who will be affected by this bug?: 1: Will affect a very few installed domains
How will those affected feel about the bug?: 3: A User would likely not purchase the product
User Pain: 0.120
Enterprise Customer affected?: Yes
School Customer affected?:
ISV affected?:
Waiting Support:
Ticket number: 2017022221000438
Bug group (optional):
Max CVSS v3 score:
hahn: Patch_Available+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Moritz Muehlenhoff univentionstaff 2011-06-09 11:59:48 CEST
Aufgetreten in der internen Umgebung: Die Festplatte des lokalen Speicherbereichs was "vollgelaufen". Qemu pausiert dann die Vms intern, in UVMM werden sie aber weiterhin als "laufend" angezeigt. Philipp sagte, dass man den Qemu-internen Zustand mit

 sudo virsh qemu-monitor-command "$VM" "info status"

abfragen kann. Dort steht nach dem Volllaufen einer Festplatte dann "paused". Mit 

 sudo virsh qemu-monitor-command "$VM" "c"

kann eine VM dann erneut gestartet werden. 

Der Qemu-Pausenzustand sollte mit dem UVMM-Zustand der VM abgeglichen werden (oder UVMM sollte den Qemu-Zustand regelmässig pollen), in der aktuellen Version wirkt das, als ob die virtualisierte Maschine gecrasht sei.
Comment 1 Philipp Hahn univentionstaff 2011-10-28 20:23:20 CEST
Das ist jetzt schon mehrfach intern aufgetreten.
Hier sollte entweder ein Skript vor dem Vollaufen warnen oder gar in UVMM der Host als problembehaftet angezeigt werden.

Hilfreich wäre auch eine Anzeige der Storage-Pool-Auslastungen (Plural!) in der Übersichtsseite der Hosts.
Comment 2 Stefan Gohmann univentionstaff 2013-04-03 21:22:48 CEST

*** This bug has been marked as a duplicate of bug 22714 ***
Comment 3 Philipp Hahn univentionstaff 2017-03-01 17:19:08 CET
Nowadays libvirt-python/examples/event-test.py provides more events and reports ENOSPC if a VM gets paused due to out-of-disk-space. The error condition is not logged otherwise by QEMU or libvirt; the management system (UVMM) must register a call-back to get libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR and libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON or libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE events and handle them:
- show the VM as paused by error
- probably log an event or escalate through some monitoring system

Monitoring free space like Bug #22714 would also be good, but is no suitable replacement for active notification.
Comment 5 Philipp Hahn univentionstaff 2018-11-28 11:32:36 CET
Merged into 4.3-2:

[4.3-2] 6bcb71e282 Bug #47956 uvmm: Fix ucslint.overrides
 .../debian/ucslint.overrides                                          | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

[4.3-2] 59932f67f3 Bug #22695: Merge branch 'phahn/22695-uvmm-enospc' into 4.3-2
[4.3-2] dcad512392 Bug #22695 uvmm: Add IO error message to frontend
 .../icons/scalable/uvmm-domain-paused-error.svg    | 882 +++++++++++++++++++++
 .../umc/js/uvmm.js                                 |  17 +-
 .../umc/python/uvmm/domains.py                     |   1 +
 3 files changed, 899 insertions(+), 1 deletion(-)

[4.3-2] 75bf8e5572 Bug #22695 uvmm: Report IO errors
 .../univention-virtual-machine-manager-daemon.yaml |  5 +--
 .../debian/changelog                               |  1 +
 .../src/univention/uvmm/node.py                    | 38 +++++++++++++++++++++-
 .../src/univention/uvmm/protocol.py                |  1 +
 4 files changed, 42 insertions(+), 3 deletions(-)

[4.3-2] 1f83a0d908 Bug #22695 uvmm: Switch to SVG images
 .../Makefile                                         |  19 -------------------
 .../umc/icons/16x16/udm-uvmm-cloudconnection.png     | Bin 642 -> 0 bytes
 .../umc/icons/16x16/udm-uvmm-cloudtype.png           | Bin 642 -> 0 bytes
 .../umc/icons/16x16/udm-uvmm-profile.png             | Bin 390 -> 0 bytes
 .../umc/icons/16x16/uvmm-cloud-off.png               | Bin 778 -> 0 bytes
 .../umc/icons/16x16/uvmm-cloud.png                   | Bin 642 -> 0 bytes
 .../umc/icons/16x16/uvmm-domain-off.png              | Bin 940 -> 0 bytes
 .../umc/icons/16x16/uvmm-domain-on.png               | Bin 742 -> 0 bytes
 .../umc/icons/16x16/uvmm-domain-paused.png           | Bin 713 -> 0 bytes
 .../umc/icons/16x16/uvmm-domain.png                  | Bin 532 -> 0 bytes
 .../umc/icons/16x16/uvmm-group.png                   | Bin 429 -> 0 bytes
 .../umc/icons/16x16/uvmm-instance-off.png            | Bin 940 -> 0 bytes
 .../umc/icons/16x16/uvmm-instance-on.png             | Bin 742 -> 0 bytes
 .../umc/icons/16x16/uvmm-instance-paused.png         | Bin 713 -> 0 bytes
 .../umc/icons/16x16/uvmm-instance-pending.png        | Bin 808 -> 0 bytes
 .../umc/icons/16x16/uvmm-instance-terminated.png     | Bin 751 -> 0 bytes
 .../umc/icons/16x16/uvmm-instance.png                | Bin 532 -> 0 bytes
 .../umc/icons/16x16/uvmm-node-off.png                | Bin 898 -> 0 bytes
 .../umc/icons/16x16/uvmm-node-qemu-off.png           | Bin 901 -> 0 bytes
 .../umc/icons/16x16/uvmm-node-qemu.png               | Bin 494 -> 0 bytes
 .../umc/icons/16x16/uvmm-node.png                    | Bin 586 -> 0 bytes
 .../umc/icons/16x16/uvmm-warn.png                    | Bin 576 -> 0 bytes
 .../umc/icons/50x50/udm-uvmm-cloudconnection.png     | Bin 2368 -> 0 bytes
 .../umc/icons/50x50/udm-uvmm-cloudtype.png           | Bin 2368 -> 0 bytes
 .../umc/icons/50x50/udm-uvmm-profile.png             | Bin 1151 -> 0 bytes
 .../umc/icons/50x50/uvmm-cloud-off.png               | Bin 3686 -> 0 bytes
 .../umc/icons/50x50/uvmm-cloud.png                   | Bin 2368 -> 0 bytes
 .../umc/icons/50x50/uvmm-domain-on.png               | Bin 2740 -> 0 bytes
 .../umc/icons/50x50/uvmm-domain-paused.png           | Bin 2338 -> 0 bytes
 .../umc/icons/50x50/uvmm-domain.png                  | Bin 1469 -> 0 bytes
 .../umc/icons/50x50/uvmm-group.png                   | Bin 1126 -> 0 bytes
 .../umc/icons/50x50/uvmm-node-off.png                | Bin 3914 -> 0 bytes
 .../umc/icons/50x50/uvmm-node-qemu-off.png           | Bin 3968 -> 0 bytes
 .../umc/icons/50x50/uvmm-node-qemu.png               | Bin 1390 -> 0 bytes
 .../umc/icons/50x50/uvmm-node.png                    | Bin 1656 -> 0 bytes
 .../umc/icons/50x50/uvmm-warn.png                    | Bin 2597 -> 0 bytes
 .../umc/js/uvmm.js                                   |   4 ++--
 .../umc/js/uvmm/DriveWizard.js                       |   2 +-

Package: univention-virtual-machine-manager-daemon
Version: 7.0.0-14A~4.3.0.201811281123
Version: 7.0.0-14A~4.3.0.201811281129
Branch: ucs_4.3-0
Scope: errata4.3-2

[4.3-2] 2cc8ae5f13 Bug #47956: univention-virtual-machine-manager-daemon 7.0.0-14A~4.3.0.201811281129
 doc/errata/staging/univention-virtual-machine-manager-daemon.yaml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 6 Erik Damrose univentionstaff 2018-11-28 16:08:49 CET
Verified: Warning and separate icon shown in UMC UVMM module when the 'outside' storage has no space left
Comment 7 Arvid Requate univentionstaff 2018-12-05 14:38:53 CET
<http://errata.software-univention.de/ucs/4.3/346.html>