Advanced networking configuration ================================= UCS-3.1 supports advanced network configurations using bridging, bonding and virtual networks (VLAN). * Bridging is often used with virtualization to connect multiple virtual machines running on a host through one shared physical network interface. * Bondings allows fail-over when a host has multiple physical network interfaces to the same network. * VLANs can be used to separate network traffic logically while using only one (or more) physical network interface. Configuration happens through setting several UCR variables to construct a valid configuration. Currently no wizard exists for graphical configuration and each setup is explained by one example, which must be adopted to the required setup. Bridging -------- Bridging allows a physical network interface to be shared by multiple virtual machines running on a single host. Instead of using one physical interface for each virtual machine and the host itself, all hosts are connected through only one (or more) uplink, which is then split up virtually just like a physical hub or switch would do. Bridges must never form a loop, where multiple paths exists from one source host to a destination host. This is managed by the Spanning Tree Protocol (STP), which Linux only supports natively in the older not-*rapid* version. If bridging is used for virtual machines and the host is a leaf node (that is the bridge does not really connect two physical network interfaces and is not supposed to forward traffic between those networks), STP should be disables and the so called *forwarding* delay should be set to 0 to allow virtual machines to be booted from the network by PXE. ### Prerequisite ### * The *bridge-utils* package must be installed: univention-install bridge-utils * The *bridge* kernel module must be loaded: modprobe bridge ucr set kernel/modules="$(ucr get kernel/modules);bridge" * Make sure to never build a network loop using bridges! ### Setup ### * This example uses two physical interfaces (eth0, eth1), but for virtual machines using only one physical interface is perfectly fine. * The physical network interfaces must not have IP addresses configured and must not be started automatically. * Instead the bridge device (br0) gets the IP address and is used as the primary interface for all services on the host itself. * The forwarding delay of the bridge is set to 0, which instantly puts the bridge in forwarding mode when new links are connected to the virtual bridge. This is needed for virtual machines to boot from the network via PXE, because otherwise the initial DHCP packets will not reach the network. ### Configuration ### xargs ucr set <<__BRIDGE__ interfaces/eth0/type=manual interfaces/eth0/start=false interfaces/eth1/type=manual interfaces/eth1/start=false interfaces/br0/address=192.168.122.13 interfaces/br0/broadcast=192.168.122.255 interfaces/br0/netmask=255.255.255.0 interfaces/br0/network=192.168.122.0 interfaces/br0/options/1="bridge_ports eth0 eth1" interfaces/br0/options/2="bridge_fd 0" interfaces/br0/start=true interfaces/primary=br0 __BRIDGE__ Bonding ------- Bonding allows two (or more) physical network interfaces to be aggregated or to be used in fail-over scenarios. Different modes are supported by the bonding driver, but this example configures a active-passive fail-over scenario. ### Prerequisite ### * The *ifenslave-2.6* package must be installed: univention-install ifenslave-2.6 * The *bonding* kernel module must be loaded: modprobe bonding ucr set kernel/modules="$(ucr get kernel/modules);bonding" * Active-passive setups can be configured with any switch, while active-active setups and channel aggregation must be supported by the switch. ### Setup ### * The physical network interfaces (eth0, eth1) must not have IP addresses configured and must not be started automatically. * Instead the bonding device (bond0) gets the IP address and is used as the primary interface for all services on the host itself. * The bonding is configured to check the link state of the physical interfaces all 100 ms using the hardwares MMI monitor. * In case of a link failure the hosts switches the active link over to the other interface and stays there until that link fails too. ### Configuration ### xargs ucr set <<__BONDING__ interfaces/eth0/type=manual interfaces/eth0/start=false interfaces/eth1/type=manual interfaces/eth1/start=false interfaces/bond0/address=192.168.122.13 interfaces/bond0/broadcast=192.168.122.255 interfaces/bond0/netmask=255.255.255.0 interfaces/bond0/network=192.168.122.0 interfaces/bond0/options/1="bond-slaves eth0 eth1" interfaces/bond0/options/2="bond-mode 1" interfaces/bond0/options/3="mmimon 100" interfaces/bond0/options/4="bond-primary eth0 eth1" interfaces/bond0/start=true interfaces/primary=bond0 __BONDING__ ### Virtual LANs ### VLANs can be used to separate network traffic by using different networks (think broadcast-domain). Instead of using multiple physically separated networks, the network packets are tagged with a VLAN-ID, which must be configured on all switches. A linkt between two switches can either transport only (untagged) packets of a single VLAN, or can transport packets of multiple VLANs; the later case calls this a *trunk*-link. The switches are responsible for for adding and removing the tags when forwarding between trunk links and links dedicated to only one VLAN. Servers are some times connected to multiple VLANs as well: In this case they are connected though trunk links as well and multiple virtual interfaces must be configured on the host, each one with its own IP address. ### Prerequisite ### * The *vlan* package must be installed: univention-install vlan * The *8021q* kernel module must be loaded: modprobe 8021q ucr set kernel/modules="$(ucr get kernel/modules);8021q" * Switches must support 802.1q VLANs. ### Setup ### * eth0 is a trunk link, which carries tagged packages of VLAN 2 and VLAN 3. * eth0 itself receives no IP address, only the untagged virtual interfaces eth0.2 and eth0.3. * eth0.2 is used as the primary interface for all host related services. ### Configuration ### xargs ucr set <<__VLAN__ interfaces/eth0/type=manual interfaces/eth0/start=false interfaces/eth0.2/address=192.168.122.13 interfaces/eth0.2/broadcast=192.168.122.255 interfaces/eth0.2/netmask=255.255.255.0 interfaces/eth0.2/network=192.168.122.0 interfaces/eth0.2/start=true interfaces/eth0.3/address=10.200.17.1 interfaces/eth0.3/broadcast=10.200.17.255 interfaces/eth0.3/netmask=255.255.255.0 interfaces/eth0.3/network=10.200.17.0 interfaces/eth0.3/start=true interfaces/primary=eth0.2 __VLAN__ All together ------------ Bonding, bridging and VLANs can be combined to implement very flexible networks. In this example the host has three physical network interfaces. One interface (eth2) is dedicated to the host itself, while the two other interfaces (eth0, eth1) are configured for high availability by using bonding. To support virtual machines in different virtual networks, VLANs 2 and 3 are used to separate network traffic. For each VLAN a separate bridge is created, so virtual machines only see their traffic and cannot use the VLAN tools themselves to get access to different VLANs. ### Prerequisite ### * The *ifenslave-2.6*, *bridge-utils* and *vlan* packages must be installed: univention-install ifenslave-2.6 bridge-utils vlan * The *bonding*, *bridge*, and *8021q* kernel modules must be loaded: modprobe bonding modprobe bridge modprobe 8021q ucr set kernel/modules="$(ucr get kernel/modules);bonding;bridge;8021q" * Switches must support 802.1q VLANs. ### Setup ### * The order in the generated /etc/network/interfaces files is very important, which is why the order is explicitly specified in this example. Otherwise the interfaces would be sorted by their names, which is not sufficient in this complex setup. * eth0 and eth1 are trunk links, which carry tagged packages of VLAN 2 and VLAN 3. * They are bound together for fail over by building a bonding device bond0, which must be explicitly started. * VLAN 2 and VLAN 3 are configured on the host and are provided untagged on the bond0.2 and bond0.3 devices. * For each VLAN a dedicate bridge br2 and br3 is created, which only has access to that one VLAN. Virtual machines should be connected to those bridges. * The host itself used the dedicated eth2 interface, which is configured normally. ### Configuration ### xargs ucr set <<__VM__ interfaces/eth0/order=2 interfaces/eth0/type=manual interfaces/eth0/start=false interfaces/eth1/order=2 interfaces/eth1/type=manual interfaces/eth1/start=false interfaces/bond0/order=3 interfaces/bond0/type=manual interfaces/bond0/options/1="bond-slaves eth0 eth1" interfaces/bond0/options/2="bond-mode 1" interfaces/bond0/options/3="mmimon 100" interfaces/bond0/options/4="bond-primary eth0 eth1" interfaces/bond0/start=true interfaces/bond0.2/order=4 interfaces/bond0.2/type=manual interfaces/bond0.2/start=false interfaces/bond0.3/order=4 interfaces/bond0.3/type=manual interfaces/bond0.3/start=false interfaces/br2/order=5 interfaces/br2/type=manual interfaces/br2/options/1="bridge_ports bond0.2" interfaces/br2/options/2="bridge_fd 0" interfaces/br2/start=true interfaces/br3/order=5 interfaces/br3/type=manual interfaces/br3/options/1="bridge_ports bond0.3" interfaces/br3/options/2="bridge_fd 0" interfaces/br3/start=true interfaces/eth2/order=1 interfaces/eth2/address=192.168.122.13 interfaces/eth2/broadcast=192.168.122.255 interfaces/eth2/netmask=255.255.255.0 interfaces/eth2/network=192.168.122.0 interfaces/eth2/start=true interfaces/primary=eth2 __VM__ Trouble shooting ---------------- * Changing the network configuration on a running system is dangerous, since any misconfiguration can make the host unreachable. Make sure to have an out-of-band access path to a root shell! * The name of an interface depends on the loading order of kernel modules and on the timing the hardware needs to reach a ready state. *udev* tries to assign persistent names to the interfaces using their MAC address, which is stored in */etc/udev/rules.d/70-persistent-net.rules*. If interfaces get removed or replaced by other interfaces with a different MAC address, old named won't get reused until that file is reset. * UVMM normally uses eth0 to configure the bridge to connect virtual machines to a network. This was archived by renaming the physical interface eth0 to peth0 and creating a bridge called eth0, into which the physical interface was connected. This is no longer recommended and automatic support for this will be removed in future versions of UCS. You can and should deactivate the scrips by setting the following UCR variables: xargs ucr set <<__UVMM__ uvmm/kvm/bridge/autostart=no uvmm/kvm/bridge/interface= xen/bridge/interface=none __UVMM__