Xen Windows GplPv

Needs Review Important page: Some parts of page are out-of-date and needs to be reviewed and corrected!

About

These drivers allow Windows to make use of the network and block backend drivers in Dom0, instead of the virtual PCI devices provided by QEMU. This gives Windows a substantial performance boost, and most of the testing that has been done confirms that. This document refers to the new WDM version of the drivers, not the previous WDF version. Some information may apply though.

I was able to see a network performance improvement of 221mbit/sec to 998mb/sec using iperf to test throughput. Disk IO, testing via crystalmark, improved from 80MB/sec to 150MB/sec on 512-byte sequential writes and 180MB/sec read performance.

With the launch of new Xen project pages the main PV driver page on www.xenproject.org keeps a lot of the more current information regarding the paravirtualization drivers.

Supported Xen versions

Gplpv >=0.11.0.213 were tested for a long time on Xen 4.0.x and are working, should also be working on Xen 4.1.

Gplpv >=0.11.0.357 tested and working on Xen 4.2 and Xen 4.3 unstable.

05/01/14 Update:

The signed drivers from ejbdigital work great on Xen 4.4.0. If you experience a bluescreen while installing these drivers, or after a reboot after installing them, please try adding device_model_version = "qemu-xen-traditional". I had an existing 2008 R2 x64 system that consitently failed with a BSOD after the gpl_pv installation. Switching to the 'qemu-xen-traditional' device model resolved the issue. However, on a clean 2008 R2 x64 system, I did not have to make this change, so please bear this in mind if you run into trouble.

I do need to de-select "Copy Network Settings" during a custom install of gpl_pv. Leaving "Copy network settings" resulted in a BSOD for me in 2008R2 x64.

I run Xen 4.4.0-RELEASE built from source on Debian Jessie amd64.

PV drivers 1.0.1089 tested on windows 7 x64 pro SP1, dom0 Debian Wheezy with xen 4.4 from source and upstream qemu >=1.6.1 <=2.0.

Notes: - upstream qemu version 1.6.0 always and older versions in some cases have critical problem with hvm domUs not related to PV drivers. - if there are domUs disks performance problem using blktap2 disks is not PV drivers problem, remove blktap2 use qdisk of upstream qemu instead for big disks performance increase (mainly in write operations)

Supported Windows versions

In theory the drivers should work on any version of Windows supported by Xen. With their respective installer Windows 2000 and later to Windows 7, 32 and 64-bit, also server versions. Please see the release notes with any version of gpl_pv you may download to ensure compatibility.

I have personally used gpl_pv on Windows 7 Pro x64, Windows Server 2008 x64, Windows Server 2008 R2 x64 and had success.

Recently I gave Windows 10 a try under Xen 4.4.1 (using Debian Jessie). The paravirtualization drivers still work. The drivers have not been installed from scratch but have been kept during the Windows Upgrade from Windows 7 to Windows 10.

Building

Sources are now available from the Xen project master git repository:

• PV xenbus driver
• PV xen interface driver
• PV network device driver
• PV storage host adapter driver
• PV network class driver

In addition you will need the Microsoft tools as described in the README files. The information under "Xen Windows GplPv/Building" still refers to the old Mercurial source code repository and is probably dated.

Downloading

New, Signed, GPL_PV drivers are available at what appears to be the new home of GPL_PV at http://www.ejbdigital.com.au/gplpv

Note: The ejbdigital site appears to be offline as of May 2015.  It appears this site may be a mirror

These may be better than anything currently available from meadowcourt or univention.

Older binaries, and latest source code, are available from http://www.meadowcourt.org/downloads/

• There is now one download per platform/architecture, named as follows:

gplpv <platform><arch> <version> <debug>.msi

• platform is '2000' for 2000, 'XP' for XP, '2003' for 2003, and 'Vista2008' for Vista/2008/7
• arch is 'x32' for 32 bit and 'x64' for 64 bits
• 'debug' if is build which contains debug info (please use these if you want any assistance in fixing bugs)
• without 'debug' build which contains no debug info

Signed drivers

Newer, signed, GPL_PV drivers are available at what appears to be the new home of GPL_PV at http://www.ejbdigital.com.au/gplpv

NOTE: The ejbdigital site appears to be offline as of May 2015.  It appears this site may be a mirror

You can get older, signed, GPLPV drivers from univention.
Signed drivers allow installation on Windows Vista and above (Windows 7, Windows Server 2008, Windows 8, Windows Server 2012) without activating the testsigning.

Installing / Upgrading

Once built (or downloaded for a binary release), the included NSIS installer should take care of everything. See here for more info, including info on bcdedit under Windows 2008 / Vista.

/!\ Please definitly visit the link above which links to /Installing . It holds information to not crash your Installation. It concerns the use of the /GPLPV boot parameter.

Using

Previous to 0.9.12-pre9, '/GPLPV' needed to be specified in your boot.ini file to activate the PV drivers. As of 0.9.12-pre9, /NOGPLPV in boot.ini will disable the drivers, as will booting into safe mode. With 'shutdownmon' running, 'xm shutdown' and 'xm reboot' issued from Dom0 should do the right thing too.

In your machine configuration, make sure you don't use the ioemu network driver. Instead, use a line like:

• vif = []

Also fixed MAC address can be set,useful to the risk of reactivation of a license for Windows.

Known Issues

This is a list of issues that may affect you, or may not. These are not confirmed issues that will consistently repeat themselves. An issue listed here should not cause you to not try gpl_pv in a safe environment. Please report both successes and failures to the mailing list, it all helps!

• An OpenSolaris Dom0 is reported not to work, for reasons unknown.
• Checksum offload has been reported to not work correctly in some circumstances.
• Shutdown monitor service in some cases is not added, and must be added manually.
• Network is not working after restore with upstream qemu, workaround for now is set fixed mac address in domUs xl cfg file.
• Installing with "Copy Network Settings" may result in a blue screen.
• A blue screen may result if you are not using the traditional qemu emulator.

PLEASE TEST YOUR PERFORMANCE USING IPERF AND/OR CRYSTALMARK BEFORE ASSUMING THERE IS A PROBLEM WITH GPL_PV ITSELF

Note: I was using pscp to copy a large file from another machine to a Windows 2008 R2 DomU machine and was routinely only seeing 12-13MB/sec download rate. I consistently had blamed windows and gpl_pv as the cause of this. I was wrong! Testing the network interface with iperf showed a substantial improvement after installing gpl_pv and the disk IO showed great performance when tested with CrystlMark. I was seeing a bug in pscp itself. Please try to test performance in a multitude of ways before submitting a complaint or bug report.

Using the windows debugger under Xen

Set up Dom0

1. Change/add the serial line to your Windows DomU config to say serial='pty'
2. Add a line to /etc/services that says 'windbg_domU 4440/tcp'. Change the domU bit to the name of your windows domain.
3. Add a line to /etc/inetd.conf that says 'windbg_domU stream tcp nowait root /usr/sbin/tcpd xm console domU'. Change the domU bit to the name of your domain. (if you don't have an inetd.conf then you'll have to figure it out yourself... basically we just need a connection to port 4440 to connect to the console on your DomU)
4. Restart inetd.

Set up the machine you will be debugging on - another Windows machine that can connect to your Dom0 from.

1. Download the windows debugger from Microsoft and install.
2. Download the 'HW Virtual Serial Port' application from HW Group and install. Version 3 appears to be out, but i've only ever used 2.5.8.

Boot your DomU

1. xm create DomU (or whatever you normally use to start your DomU)
2. Press F8 when you get to the windows text boot menu and select debugging mode, then boot. The system should appear to hang before the splash screen starts

Start HWVSP

1. Start the HW Virtual Serial Port application
2. Put the IP address or hostname of your Dom0 in under 'IP Address'
3. Put 4440 as the Port
4. Select an unused COM port under 'Port Name' (I just use Com8)
5. Make sure 'NVT Enable' in the settings tab is unticked
6. Save your settings
7. Click 'Create COM'. If all goes well it shuold say 'Virtual serial port COM8 created' and 'Connected device <hostname>'

Run the debugger

1. Start windbg on your other windows machine
2. Select 'Kernel Debug' from the 'File' menu
3. Select the COM tab, put 115200 in as the baud rate, and com8 as the port. Leave 'Pipe' and 'Reconnect' unticked
4. Click OK
5. If all goes well, you should see some activity, and the HWVSP counters should be increasing. If nothing happens, or if the counters start moving and then stop, quit windbg, delete the com port, and start again from 'Start HWVSP'. Not sure why but it doesn't always work the first time.

Debugging

1. The debug output from the PV drivers should fly by. If something isn't working, that will be useful when posting bug reports.
2. If you actually want to do some debugging, you'll need to have built the drivers yourself so you have the src and pdb files. In the Symbol path, add '*SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols;c:\path_to_source\target\winxp\i386'. change winxp\i386 to whatever version you are debugging.
3. Actually using the debugger is beyond the scope of this wiki page :)

Developers

• xenpci driver - communicates with Dom0 and implements the xenbus and event channel interfaces
• xenhide driver - disables the QEMU PCI ATA and network devices when the PV devices are active
• xenvbd driver - block device driver
• xennet driver - network interface driver
• xenstub driver - provides a dummy driver for vfb and console devices enumerated by xenpci so that they don't keep asking for drivers to be provided.