Bug 47701

Summary: Automatically reconnect VMs to bridge (4.4)
Product: UCS Reporter: Philipp Hahn <hahn>
Component: Virtualization - KVMAssignee: Philipp Hahn <hahn>
Status: CLOSED FIXED QA Contact: Erik Damrose <damrose>
Severity: normal    
Priority: P5 CC: damrose, heidelberger, oliver.bohlen, voelker
Version: UCS 4.4   
Target Milestone: UCS 4.4-0-errata   
Hardware: Other   
OS: Linux   
What kind of report is it?: Bug Report What type of bug is this?: 5: Major Usability: Impairs usability in key scenarios
Who will be affected by this bug?: 3: Will affect average number of installed domains How will those affected feel about the bug?: 5: Blocking further progress on the daily work
User Pain: 0.429 Enterprise Customer affected?: Yes
School Customer affected?: ISV affected?:
Waiting Support: Flags outvoted (downgraded) after PO Review:
Ticket number: 2019060521000629 Bug group (optional):
Max CVSS v3 score:
Bug Depends on:    
Bug Blocks: 49607, 49604, 49606    

Description Philipp Hahn univentionstaff 2018-08-30 17:26:57 CEST
If any UCRV interfaces/* changes, the network is re-configured and restarted.
With Qemu/KVM this removes and re-adds the bridge.
All VMs will loose their network connection until /usr/lib/univention-virtual-machine-manager-node-kvm/ucs-kvm-reconnect-bridge is manually invoked.

UCS should do this automatically:
 ucr set interfaces/br0/options/65535='post-up /usr/lib/univention-virtual-machine-manager-node-kvm/ucs-kvm-reconnect-bridge -b $IFACE'
Comment 1 Philipp Hahn univentionstaff 2018-08-30 17:30:26 CEST
The network sometimes is also restarted during an upgrade.
Yes, we recommend to migrate VMs away to update the host, but it also has happened during the installation on an errata update.
And as UVMM / UCS has no feature to do automatically update a cluster of KVM nodes by migrating VMs away from one host, update it, and migrate VMs back before updating the next one, this is a bug.
Comment 2 Philipp Hahn univentionstaff 2018-11-05 09:52:36 CET
again 2018-11-05: phahn_piuparts1 @ sparka
Comment 3 Philipp Hahn univentionstaff 2018-11-16 18:48:00 CET
(In reply to Philipp Hahn from comment #0)
> UCS should do this automatically:
>  ucr set interfaces/br0/options/65535='post-up /usr/lib/univention-virtual-machine-manager-node-kvm/ucs-kvm-reconnect-bridge -b $IFACE'

This does not work as then the script is executed during the initial boot process when libvirtd is not yet running - libvirtd.service is started AFTER networking.service.
So the script must check that libvirtd is running and/or exit cleanly if it is not.
Comment 4 Philipp Hahn univentionstaff 2018-11-30 18:21:00 CET
again 2018-11-30: phahn_piuparts1 @ sparka
Comment 5 Philipp Hahn univentionstaff 2019-04-02 08:44:02 CEST
again 2019-04-02: phahn_piuparts1 @ sparka
Comment 6 Erik Damrose univentionstaff 2019-05-13 10:07:53 CEST
again: https://help.univention.com/t/11645
Comment 7 Philipp Hahn univentionstaff 2019-06-05 13:58:22 CEST
again at customer site during automatic errata updates: all VMs were unreachable; which included business critical VMs!
Comment 8 Philipp Hahn univentionstaff 2019-06-05 14:22:51 CEST
This bug is triggered by "univention-base-files", which does a "ucr register" in its postinst script, which includes something similar to "ucr commit" on all files and handlers provided by the package.
Comment 9 Philipp Hahn univentionstaff 2019-06-06 14:42:50 CEST
[4.4-0] 65322aef82 Bug #47701 uvmm: Re-connect VMs to bridges on network restart
 .../univention-virtual-machine-manager-node.yaml   | 10 ++++
 .../debian/changelog                               |  6 +++
 .../debian/control                                 |  1 +
 ...ivention-virtual-machine-manager-node-kvm.if-up | 55 ++++++++++++++++++++++
 4 files changed, 72 insertions(+)

Package: univention-virtual-machine-manager-node
Version: 7.0.1-4A~4.4.0.201906061438
Branch: ucs_4.4-0
Scope: errata4.4-0

[4.4-0] 87e66ab613 Bug #47701: univention-virtual-machine-manager-node 7.0.1-4A~4.4.0.201906061438
 doc/errata/staging/univention-virtual-machine-manager-node.yaml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

QA:
1. ifdown -a ; ifup -a: All VMs are re-connected
2. apt install univention-base-{files,packages}: dito
3. reboot: network is started, afterwards libvirt starts VMs.
Comment 10 Erik Damrose univentionstaff 2019-06-07 10:08:36 CEST
b0cc9f6d update yaml, fix version

OK: /etc/network/if-up.d/univention-virtual-machine-manager-node-kvm
OK: use existing reconnect-kvm-bridge script
OK: executed per interface, if it is a bridge
OK: only run and try to connect when libvirt is running (interfaces are queried from libvirt)
OK: reconnect works for different scenarios,
 * ucr set interface/<name>/<something>
 * u-base-files package update
 * ifdown + ifup
OK: Network on restarted VMs after hypervisor reboot
OK: yaml
Verified
Comment 11 Erik Damrose univentionstaff 2019-06-07 13:24:22 CEST
<http://errata.software-univention.de/ucs/4.4/143.html>