…nerdy by nature!

For one of my virtual servers running under Xen in a routed setup, I was required to have more than one external interface, meaning the VM should be available on the net on multiple IP addresses.

First, I tried to assign multiple interfaces to the VM, but that didn’t really work as the kernel was confused on which route it was supposed to send out data. Turns out it’s even easier. Let’s say you have the two IPs 1.2.3.4 and 5.6.7.8 which you want to assign to your domU. First, you need to put a line like this into your domU config:

vif=["ip=1.2.3.4/32 5.6.7.8/32,mac=00:1D:92:AA:BB:CC"]

Within the domU, you would then create an interface (eth0) with a static configuration for the IP 1.2.3.4. Afterwards, you would simply create an alias with the second ip:

ifconfig eth0:1 5.6.7.8 up

Now you should be able to reach your domU on both interfaces.

Today, I had the mindnumbing task of installing openSUSE (current version 11.3) as a virtual Xen guest (domU) within a CentOS 5 dom0. There are, of course, multiple solutions to this problem and the most convenient is probably the one found here.

However, my setup is a little bit different as – thanks to my hoster’s secure routing – all my domUs need to route their traffic through the main IP of the dom0, thus preventing me from setting the right network information (netmask 255.255.255.255 and a point-to-point route to the default gw) in YaST installation.

As a solution to the problem, I first installed openSUSE as a HVM domain from a downloaded DVD-ISO. Afterwards, I made some small changes and ran the system in paravirtualized mode. In order to do so, you need twice as much space as the installation takes, since you need to copy the data from the HVM disk to the paravirtualized disk.

So for this example, let’s say you have a volume group named opensuse with two logical volumes disk1 and disk2, where you start the HVM on disk2 and disk1 will be your root device for the paravirtualized machine. You’ll also need to download the openSUSE DVD image in ISO format.

Create a HVM domain

First of all, we need a HVM enabled config. Be sure to enable PAE, otherwise you’ll get an error when booting from the DVD. Here’s the config I used:

name="opensuse1"
memory=768
vcpus=1
on_poweroff="destroy"
on_reboot="restart"
on_crash="restart"
disk=["phy:/dev/opensuse/disk2,xvda,w", "file:/path/to/openSUSE-11.3-DVD-x86_64.iso,xvdb:cdrom,r"]
kernel="/usr/lib/xen/boot/hvmloader"
builder="hvm"
device_model ="/usr/lib64/xen/bin/qemu-dm"
boot="d"
sdl=0
vnc=1
vncconsole=1
usbdevice="tablet"
shadowmemory=8
pae=1
acpi=0

Of course, you need to adapt the path to your local setting. If you install the domU on a remote host, make sure you’ve got X11 forwarding enabled, otherwise the VNC window will not pop up. You’ll be greeted with the standard openSUSE boot promt, so you can just carry out the installation as you normally. Just be sure to include the package kernel-xen in the installation. And if you’re smart, you’ll only create a single partition (no swap etc) on which the system gets installed, saving you the hassle of copying data from several partitions.

Copying data from HVM disk to paravirtualized disk

Once the domU has been set up, shut it down. Now, you need to copy all files from the HVM disk (/dev/opensuse/disk2) to the disk designated for the paravirtualized domain (/dev/opensuse/disk1). If you haven’t done this yet, put a file system of your choice on that disk, e.g.

mkfs.ext3 /dev/opensuse/disk1

Since the HVM domain uses a complete disk structure (including MBR etc) instead of a single partition, you need to mount only a single partition out of the disk structure. You can either specify an offset in the mount command or, if you’re a little lazy, use kpartx:

kpartx -av /dev/opensuse/disk2

This will create mappings for all partitions found in /dev/opensuse/disk2. If you only created one partition as I suggested, you’ll find the mapping at /dev/mapper/opensuse-disk21.

Now, mount the two disks to seperate mount points and begin copying:

mkdir /mnt/disk1
mkdir /mnt/disk2
mount /dev/opensuse/disk1 /mnt/disk1
mount /dev/mapper/opensuse-disk21 /mnt/disk2
cp -ax /mnt/disk2/* /mnt/disk1

After the process has completed, you need to make some small changes.

Change the file /mnt/disk1/boot/grub/menu.lst to boot the Xen enabled kernel. It only needs one entry like this:

default 0
timeout 0

title Xen
  root (hd0,0)
  kernel /boot/vmlinuz-xen root=/dev/hda1 splash=none showopts vga=0x314 console=xvc0
  initrd /boot/vmlinuz-xen

Make sure the kernel and the initrd exist. The console parameter is also important, otherwise you won’t see anything on the Xen console. We also need to make sure to enable this console in two more files.

Edit /mnt/disk1/etc/inittab and change the following line

1:2345:respawn:/sbin/mingetty --noclear tty1

to look like this:

1:2345:respawn:/sbin/mingetty --noclear xvc0

Now edit /mnt/disk1/etc/securetty and append xvc0 to the list. The last change isin /mnt/disk1/etc/fstab: Make sure you set the root device to the one from the domU config (e.g. /dev/hda1), as YaST will have put something like /dev/disk-by-id… in there.

Afterwards, you can umount the two disks:

umount /mnt/disk1
umount /mnt/disk2

Starting in paravirtualized mode

You still need to adapt the domU’s config to start in paravirtualized mode. Here’s what my config reads like:

name="opensuse1"
memory=768
vcpus=1
on_poweroff="destroy"
on_reboot="restart"
on_crash="restart"
disk=["phy:/dev/opensuse/disk1,hda1,w"]
bootloader="/usr/bin/pygrub"
vif=["ip=192.168.1.1"]

The network setting looks slightly different, but you need to change this file anyway

Now, you can simply start the domU and it will get created as a paravirtualized domain.

Cleanup & Conclusion

If you don’t plan on restarting the dom0 in the next days, you should clear the mapping kpartx created by invoking:

kpartx -dv /dev/opensuse/disk2

Also, you are now free to either remove disk2 or add it to the paravirtualized machine for additional space.

I know the solution described in this post is not the most elegant one, but it worked for me right now. Hope it helps someone else, too…

After installing a Windows XP under KVM today, I was wondering if there were any guest tools available for KVM, like virtually every other virtualization (VMware, VBox, even Xen) system provides them. The good news is, there are drivers available, at least for speeding up network and harddisk access, but none for graphics though.

The bad news for me, though, was the fact, that the harddisk driver should be added during Windows installation, and not afterwards. But with a little trick, switching the driver after installation works as well. This is archived by adding a second temporary disk to the setup, which uses the KVM drivers.

So in detail, consider your KVM command line, which should contain something similar to this:

-drive file=WinXp.qcow,if=ide,boot=on

The if=ide part tells qemu to use the standard IDE drivers, which emulate an Intel IDE controller. A simple switch to if=virtio would result in an unbootable machine, as Windows would have no drivers to access the disk.

So for the workaround, we are going to create another (small) disk and inject it. First of all, you need to download the viostor drivers, either from the original homepage or (better still for our needs) packed as an ISO file.

Now, put the ISO into the folder that contains the disk file(s) for your KVM machine, open a terminal in that directory and type:

# qemu-img create temp.qcow 1MB

This will create an image of 1MB with the name temp.qcow. Now assuming that the ISO you downloaded is named NetKVM-and-viostor.iso, type in your regular KVM command to fire up the virtual machine and append the following:

-drive file=temp.qcow,if=virtio,boot=off -cdrom NetKVM-and-viostor.iso

This will add another harddrive to your machine, which requires the viostor drivers. Also, the ISO containing the needed drivers is inserted as a CD drive.

Once the machine has started, Windows will query you for drivers needed to communicate with the new disk. Install the drivers from the CD and shutdown the machine. Now you can delete the temporary image, remove the added parameters from above and replace the parameter containing your hard disk with

-drive file=WinXp.qcow,if=virtio,boot=on

As Windows has already installed the drivers, it will know how to communicate with the disk. From now on, disk access should be faster.

Oh, by the way, changing the network interface to use virtio drivers is a lot easier, you just need to replace

-net nic

in your command with

-net nic,model=virtio

Again, Windows will ask you for drivers, since they are in the same ISO you can just as well install them while you’ve got the CD mounted.