Using IVS with libvirt (KVM)

Prerequisites

Follow the instructions at the IVS Installation Guide to get IVS installed and running.

Installation

Install the packages required for kvm

sudo apt-get install kvm libvirt-bin virtinst

Remove the bridge created by libvirt
(Do this step only if you are exclusively using xenon for your virtual network topology on this host.)

sudo virsh net-destroy default; sudo virsh net-autostart \--disable default

Create the following script (call it "/etc/ivs-ifup") which adds the TAP interface to the xenon switch when machines are started

#!/bin/sh
/sbin/ifconfig $1 0.0.0.0 up
/usr/sbin/xenon-ctl add-port indigo $1

Make the configuration scripts executable

sudo chmod+x /etc/ivs-ifup

Add the following configuration snippet to the bottom of "/etc/apparmor.d/abstractions/libvirt-qemu" to configure apparmor to allow the necessary network configuration changes when virtual machines are started/stopped.

/etc/apparmor.d/abstractions/libvirt-qemu

# needed to make network changes
capability net_admin,

#network configuration scripts/binaries
/usr/sbin/ivs-ctl rmix,
/sbin/ifconfig rmix,
/etc/ivs-ifup rmix,

Reload apparmor to apply the configuration changes.

sudo /etc/init.d/apparmor reload

Add the following configuration snippet to the bottom of /etc/libvirt/qemu.conf to allow libvirt to make networking changes.

user = "root"
group = "root"

cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "/dev/hpet","/dev/net/tun",
]

Restart libvirt-bin to apply the changes

sudo /etc/init.d/libvirt-bin restart

Configure any virtual machines with the following interface statement under <devices> to attach them to the IVS switch.

sudo virsh edit <vmname>

<interface type='ethernet'>
  <script path='/etc/ivs-ifup'/>
</interface>

Here is a full sample of an XML configuration for a VM running on a IVS switch.

<domain type='kvm'>
  <name>newvm2</name>
  <uuid>cd4e4b03-487d-6583-0b95-d4a41cc2503b</uuid>
  <memory>262144</memory>
  <currentMemory>262144</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/administrator/newvm2/tmpRVK6eg.qcow2'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='ethernet'>
      <mac address='52:54:00:d6:48:b1'/>
      <script path='/etc/ivs-ifup'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </memballoon>
  </devices>
</domain>

After starting your virtual machines, you should be able to see the tap interfaces connected to the switch with ivs-ctl show.

~$ ivs-ctl show
indigo:
  kernel lookups: hit=0 missed=390 lost=12
  kernel flows=0
  ports:
    0 indigo (internal)
      rx: packets=12 bytes=720 errors=0 dropped=0
      tx: packets=0 bytes=0 errors=0 dropped=0
    1 tap0
      rx: packets=35 bytes=4213 errors=0 dropped=0
      tx: packets=35 bytes=4402 errors=0 dropped=0
    2 tap1
      rx: packets=35 bytes=4402 errors=0 dropped=0
      tx: packets=35 bytes=4213 errors=0 dropped=0
    1023 gre (gre)
      rx: packets=0 bytes=0 errors=0 dropped=0
      tx: packets=0 bytes=0 errors=0 dropped=362