Xen FAQ Linux, Running on

A large warning message is displayed when I boot Linux, saying something about '/lib/tls'. What should I do?

Do as the message advises! Some modern distributions ship with a 'TLS' version of glibc that is not fully compatible with Xen. To use Xen reliably and with maximum performance you must disable the incompatible glibc. You can easily do this by moving the /lib/tls directory (e.g., mv /lib/tls /lib/tls.disabled).

Alternatively, install a Xen-specific GLIBC, see XenSpecificGlibc.

Lots of programs crash with 'Segmentation fault' when I boot my system. What's going wrong?

See the answer to the previous question. In some environments (for example, when running Xen within a VMware product) you may experience severe problems with TLS libraries. If these are so severe that you cannot even log in to your system, you can obtain a command prompt by temporarily adding the boot parameter init=/bin/bash to your Linux command line.

I can not export block device as /dev/sdaX in DomU

You have errors like

Registering block device major 8
register_blkdev: cannot get major 8 for sd
xen_blk: can't get major 8 with name sd

all you need to do is disable the entire scsi subsystem, ie:

CONFIG_SCSI=n

This allows the xen block frontend to register on device ID's normally occupied by the scsi subsystem.

When I start a 9th domain using a loop-back file as its virtual disk why do I get "Error: vbd: Segment not found: uname=file:/path/to/image"?

By default, Linux only allows up to 8 loop-back devices to be active. You can increase this number by setting max_loop=32 (etc) on the domain 0 kernel command line in Grub. You also need to create new loop[n] device nodes in /dev.

I'm trying to get X working in domain 0 but the server seems to hang (or domain 0 panics). What's going on?

It's likely your X server is trying to use agpgart, support for which is currently included only in the 2.0-testing and unstable trees. Try downloading a new tarball, or clone one of the BK repositories.

I get "Badness in local_bh_enable at kernel/softirq.c" messages, why is this?

This is fairly likely to be caused by a module compiled for native i386 rather than Xen. When building modules outside of the Xen build tree, use make ARCH=xen ....

Alternative, this may be a driver that uses interrupt en/disabling instructions directly rather than the proper API....

How do I load and run an initial ram disk (initrd)?

To specify an initrd file for domain 0 add an extra "module" line to your GRUB config, after your kernel-module line (e.g., module /boot/initrd-2.6-xen0.img). To use an initrd file for other domains, set the "ramdisk" variable in your domain configuration file (e.g., ramdisk = path/to/initrd-2.6-xenU.img).

The initrd file that came with my distro doesn't work well with Xen. How do I edit it?

There are various forms of initrd image, but the two most common formats are a gzip'ed filesystem and a gzip'ed cpio file. In either case you can take the following steps to modify a file initrd.img (when running as root):

# mkdir initrd.mnt
# gzip -d -S ".img" /path/to/initrd.img
# mount -o loop /path/to/initrd initrd.mnt OR (cd initrd.mnt && cpio -ic <../path/to/initrd)
# ...edit files in directory initrd.mnt....
# umount initrd.mnt OR (cd initrd.mnt && find . | cpio -oc --quiet >../path/to/initrd)
# gzip -9 -S ".img" /path/to/initrd

If you use cpio then you may get warnings about truncated inode numbers. These can be ignored. Some distros (notably SLES 9) have a mkinitrd that generates initrd files with garbage at the end of them. Xen will not load this initrds. You can work around this problem by gunzip'ing and gzip'ing the initrd again.

When I use the balloon driver, why do tools like 'top' and 'free' not show a change in memory usage?

The balloon driver requests memory from the system allocator which it then directly gives back to Xen. From the kernel's perspective, however, that memory is still part of the system and is being used by the balloon driver. Hence the unintuitive memory statistics.

I increased the memory (RAM) in a domain and rebooted it, but it still shows the old memory value?

You need to "re-create" the domain using the

xm create <domU>

command again. Simply rebooting the domain does not cause it to read the new value from the domain definition file.

Booting a guest complains about trying to use an already-mounted block device, but it isn't mounted?!

This is a bug to do with Xen incorrectly freeing guest resources. Take a look at the last line in /var/log/xen-hotplug.log, something like:

xenstore-read: couldn't read path /local/domain/9/vm

Take a note of that number, and run xenstore-rm backend/vbd/9, replacing 9 with the number you read. The guest machine should then boot happily.

Be careful when doing this that there are no guests effectively using the device in question, damage will probably be incurred in that case!