Xen Project Software Overview: Difference between revisions

From Xen
Jump to navigationJump to search
Line 133: Line 133:
* [http://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#PVH-Guest-Specific-Options PVH specific Config options] (Xen 4.10+)
* [http://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#PVH-Guest-Specific-Options PVH specific Config options] (Xen 4.10+)
* Currently PVH only supports [http://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#Direct-Kernel-Boot Direct Kernel Boot]. EFI support is currently being developed.
* Currently PVH only supports [http://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#Direct-Kernel-Boot Direct Kernel Boot]. EFI support is currently being developed.


{{Anchor|ARM}}


=== ARM Hosts ===
=== ARM Hosts ===
On ARM hosts, there is only one virtualization mode, which does not use QEMU.
On ARM hosts, there is only one virtualization mode, which does not use QEMU.

{{Anchor|Summary}}

=== Summary ===
=== Summary ===
The following diagram gives an overview of the various virtualization modes implemented in Xen.
The following diagram gives an overview of the various virtualization modes implemented in Xen.

[[File:XenModes.png|thumb|none|750px|This image gives a brief overview of the different Xen Virtualization modes]]

'''Footnotes:'''
# Uses QEMU on older hardware and hardware acceleration on newer hardware – see 3)
# Always uses Event Channels
# Implemented in software with hardware accelerator support from IO APIC and posted interrupts
# PVH uses Direct Kernel Boot or PyGrub. EFI support is currently being developed.
# PV uses PvGrub for boot
# ARM guests use EFI boot or Device Tree for embedded applications


== Toolstacks, Managment APIs and Consoles ==
== Toolstacks, Managment APIs and Consoles ==

Revision as of 21:51, 9 November 2017

Icon todo.png To Do:

The PVH section and relevant images on the right need to be re-written with a view to Xen 4.10 and PVHv2

Shortcuts
Features
Xen Project Features
Xen Project-enabled Kernels
Xen Project-Enabled operating systems
PV-Enabled operating systems
Xen Kernel Feature Matrix
Guest types
Paravirtualization (PV)
HVM
PV on HVM
PV on HVM Drivers
Information about using PV-on-HVM drivers
HowTo on PV-on-HVM drivers
Windows drivers
Toolstacks
Choice of ToolStacks
Xen Project or XCP?
Host Install
Host OS Install Considerations
Installing Hosts
Live CDs, DVDs, etc.
Guest Install
Installing Guests
Guest VM Images
v · d · e

What is the Xen Project Hypervisor?

The Xen Project hypervisor is an open-source type-1 or baremetal hypervisor, which makes it possible to run many instances of an operating system or indeed different operating systems in parallel on a single machine (or host). The Xen Project hypervisor is the only type-1 hypervisor that is available as open source. It is used as the basis for a number of different commercial and open source applications, such as: server virtualization, Infrastructure as a Service (IaaS), desktop virtualization, security applications, embedded and hardware appliances. The Xen Project hypervisor is powering the largest clouds in production today.

Here are some of the Xen Project hypervisor's key features:

  • Small footprint and interface (is around 1MB in size). Because it uses a microkernel design, with a small memory footprint and limited interface to the guest, it is more robust and secure than other hypervisors.
  • Operating system agnostic: Most installations run with Linux as the main control stack (aka "domain 0"). But a number of other operating systems can be used instead, including NetBSD and OpenSolaris.
  • Driver Isolation: The Xen Project hypervisor has the capability to allow the main device driver for a system to run inside of a virtual machine. If the driver crashes, or is compromised, the VM containing the driver can be rebooted and the driver restarted without affecting the rest of the system.
  • Paravirtualization: Fully paravirtualized guests have been optimized to run as a virtual machine. This allows the guests to run much faster than with hardware extensions (HVM). Additionally, the hypervisor can run on hardware that doesn't support virtualization extensions.


This page will explore the key aspects of the Xen Project architecture that a user needs to understsand in order to make the best choices.

  • Guest types: The Xen Project hypervisor can run fully virtualized (HVM) guests, or paravirtualized (PV) guests.
  • Domain 0: The architecture employs a special domain called domain 0 which contains drivers for the hardware, as well as the toolstack to control VMs.
  • Toolstacks: This section covers various toolstack front-ends available as part of the Xen Project stack and the implications of using each.

Introduction to Xen Project Architecture

Below is a diagram of the Xen Project architecture. The Xen Project hypervisor runs directly on the hardware and is responsible for handling CPU, Memory, and interrupts. It is the first program running after exiting the bootloader. On top of the hypervisor run a number of virtual machines. A running instance of a virtual machine is called a domain or guest. A special domain, called domain 0 contains the drivers for all the devices in the system. Domain 0 also contains a control stack to manage virtual machine creation, destruction, and configuration.

Xen Arch Diagram.png

Components in detail:

  • The Xen Project Hypervisor is an exceptionally lean (<150,000 lines of code) software layer that runs directly on the hardware and is responsible for managing CPU, memory, and interrupts. It is the first program running after the bootloader exits. The hypervisor itself has no knowledge of I/O functions such as networking and storage.
  • Guest Domains/Virtual Machines are virtualized environments, each running their own operating system and applications. The hypervisor supports two different virtualization modes: Paravirtualization (PV) and Hardware-assisted or Full Virtualization (HVM). Both guest types can be used at the same time on a single hypervisor. It is also possible to use techniques used for Paravirtualization in an HVM guest: essentially creating a continuum between PV and HVM. This approach is called PV on HVM. Guest VMs are totally isolated from the hardware: in other words, they have no privilege to access hardware or I/O functionality. Thus, they are also called unprivileged domain (or DomU).
  • The Control Domain (or Domain 0) is a specialized Virtual Machine that has special privileges like the capability to access the hardware directly, handles all access to the system’s I/O functions and interacts with the other Virtual Machines. It also exposes a control interface to the outside world, through which the system is controlled. The Xen Project hypervisor is not usable without Domain 0, which is the first VM started by the system.
  • Toolstack and Console: Domain 0 contains a control stack (also called Toolstack) that allows a user to manage virtual machine creation, destruction, and configuration. The toolstack exposes an interface that is either driven by a command line console, by a graphical interface or by a cloud orchestration stack such as OpenStack or CloudStack.
  • Xen Project-enabled operating systems: Domain 0 requires a Xen Project-enabled kernel. Paravirtualized guests require a PV-enabled kernel. Linux distributions that are based on recent Linux kernel are Xen Project-enabled and usually include packages that contain the hypervisor and Tools (the default Toolstack and Console). All but legacy Linux kernels are PV-enabled, capable of running PV guests.

Also see:

Guest Types

This section gives an overview of guest types, such that the reader can make informed decisions about which guest type to use. On ARM hosts, there is only one guest type, while on x87 hosts the hypervisor supports running of three types of guests:

  • Paravirtualized Guests or PV Guests: PV is a software virtualization technique originally introduced by Xen Project and later adopted by other virtualization platforms. PV does not require virtualization extensions from the host CPU, but requires Xen-aware guest operating systems.
  • HVM Guests: HVM guests use virtualization extensions from the host CPU to virtualize guests. HVM requires Intel VT or AMD-V hardware extensions. The Xen Project software uses QEMU device models to emulate PC hardware, including BIOS, IDE disk controller, VGA graphic adapter, USB controller, network adapter etc.
  • PVH Guests: PVH guests are lightweight HVM-like guests that use virtualization extensions from the host CPU to virtualize guests. Unlike HVM guests, PVH guests do not use QEMU to emulate devices, but use PV drivers for I/O and native operating system interfaces for virtualized timers, virtualized interrupt and boot. PVH guests require PVH enabled guest operating system. This approach is similar to how Xen virtualizes ARM guests, with the exception that ARM CPUs provide hardware support for virtualized timers and interrupts.

In implementation terms, there are two major execution paths on x86 which implement these three guest types: the PV path implements PV guests; the HVM path implements HVM and PVH guests.

Note that originally we implemented PVH (v1) using the PV path, while making use of Hardware assisted virtualization. In Xen 4.9 and Xen 4.10 we replaced PVHv1 with an alternative implementation called PVHv2 using the HVM path.

Icon Ambox.png IMPORTANT: Guest types are selected through builder configuration file option for Xen 4.9 or before and the type configuration file option from Xen 4.10 onwards in the (also see man pages).


PV (x86)

Paravirtualization (PV) is a virtualization technique originally introduced by Xen Project, later adopted by other virtualization platforms. PV does not require virtualization extensions from the host CPU and is thus ideally suited to run on older Hardware. However, paravirtualized guests require a PV-enabled kernel and PV drivers, so the guests are aware of the hypervisor and can run efficiently without emulation or virtual emulated hardware. PV-enabled kernels exist for Linux, NetBSD and FreeBSD. Linux kernels have been PV-enabled from 2.6.24 using the Linux pvops framework. In practice this means that PV will work with most Linux distributions (with the exception of very old versions of distros).

Also see:

HVM and its variants (x86)

Full Virtualization or Hardware-assisted virtualization (HVM) uses virtualization extensions from the host CPU to virtualize guests. HVM requires Intel VT or AMD-V hardware extensions. The Xen Project software uses Qemu to emulate PC hardware, including BIOS, IDE disk controller, VGA graphic adapter, USB controller, network adapter etc. Virtualization hardware extensions are used to boost performance of the emulation. Fully virtualized guests do not require any kernel support. This means that Windows operating systems can be used as a Xen Project HVM guest. For older host operating systems, fully virtualized guests are usually slower than paravirtualized guests, because of the required emulation.

To address this, the Xen Project community has upstreamed PV drivers to Linux and other open source operating systems. On operating systems with Xen Support, these drivers will be automatically used when you select the HVM virtualization mode. On Windows this requires that appropriate PV drivers are installed. You can find more information at

HVM mode, even with PV drivers, has a number of things that are unnecessarily inefficient. One example are the interrupt controllers: HVM mode provides the guest kernel with emulated interrupt controllers (APICs and IOAPICs). Each instruction that interacts with the APIC requires a trip up into Xen and a software instruction decode; and each interrupt delivered requires several of these emulations. Many of the the paravirtualized interfaces for interrupts, timers, and so on are available for guests running in HVM mode: they just need to be turned on and used. This includes Viridian (i.e. Hyper-V) enlightenments which ensure that Windows guests are aware they are virtualized, which speeds up Windows workloads running on Xen.This required some changes to operating systems with Xen Support, which when available are automatically used. When used, we commonly talk about PVHVM guests (using PVHVM or PV-on-HVM or drivers), even though these are just HVM guests.

This image gives a brief overview of the different HVM variants.

Compared to PV based virtualization, PVHVM is generally faster.

Also see:


PVH (x86)

A key motivation behind PVH is to combine the best of PV and HVM mode and to simplify the interface between operating systems with Xen Support and the Xen Hypervisor. To do this, we had two options: start with a PV guest and implement a "lightweight" HVM wrapper around it (as we have done for ARM) or start with a HVM guest and remove functionality that is not needed. The first option looked more promising based on our experience with the Xen ARM port, than the second. This is why we started developing an experimental virtualization mode called PVH (now called PVHv1) which was delivered in Xen Project 4.4 and 4.5. Unfortunately, the initial design did not simplify the operating system - hypervisor interface to the degree we hoped: thus, we started a project to evaluate the second option, which was significantly simpler. This led to PVHv2 (which in the early days was also called HVMLite). PVHv2 guests are lightweight HVM guests which use Hardware virtualization support for memory and privileged instructions, PV drivers for I/O and native operating system interfaces for everything else. PVHv2 also does not use QEMU.

PVHv1 has been replaced with PVHv2 in Xen 4.9, and has been made fully supported in Xen 4.10. PVH (v2) requires guests with Linux 4.11 or newer kernel.


Also see:


ARM Hosts

On ARM hosts, there is only one virtualization mode, which does not use QEMU.

Summary

The following diagram gives an overview of the various virtualization modes implemented in Xen.

This image gives a brief overview of the different Xen Virtualization modes

Footnotes:

  1. Uses QEMU on older hardware and hardware acceleration on newer hardware – see 3)
  2. Always uses Event Channels
  3. Implemented in software with hardware accelerator support from IO APIC and posted interrupts
  4. PVH uses Direct Kernel Boot or PyGrub. EFI support is currently being developed.
  5. PV uses PvGrub for boot
  6. ARM guests use EFI boot or Device Tree for embedded applications

Toolstacks, Managment APIs and Consoles

Xen Project software employs a number of different toolstacks. Each toolstack exposes an API, which will run different tools. The figure below gives a very brief overview of the choices you have, which commercial products use which stack and examples of hosting vendors using specific APIs.

Boxes marked in blue are developed by the Xen Project

The Xen Project software can be run with the default toolstack, with Libvirt and with XAPI. The pairing of the Xen Project hypervisor and XAPI became known as XCP which has been superceded by open source XenServer. The diagram above shows the various options: all of them have different trade-offs and are optimized for different use-cases. However in general, the more on the right of the picture you are, the more functionality will be on offer.

Which to Choose?

  • The article Choice of ToolStacks gives you an overview of the various options, with further links to tooling and stacks for a specific API exposed by that toolstack.
  • Xen or XCP also provides good pointers on whether to use Xen or XCP (which has been supplanted by open source XenServer).

Of course there are also additional management tools available for different API's. For more information see:

Xen Project:


Getting Xen Project, Host and Guest Install

Choice of Control Domain (Dom0)

As stated earlier, the Xen Project hypervisor requires a kernel as control domain. Most Xen Project-enabled kernels are very similar from the perspective of the hypervisor itself. Choosing the right Dom0 for you comes down to:

  • How familiar you are with a specific distro (e.g. packaging system, etc.)
  • Xen Project Hypervisor version that ships with the distro
  • Whether you can get commercial support (if you need it)

If you use XCP, you typically will not be interfacing much with Dom0. That is unless you are a power user.

Also See

Getting Xen Project software

The Xen Project hypervisor is available as source distribution from XenProject.org. However, you can get recent binaries as packages from many Linux and Unix distributions, both open source and commercial.

Xen Project Source Distributions The Xen Project community delivers the hypervisor as a source distribution, following the delivery model of the Linux kernel. The software is released approximately once every 6-9 months, with several update releases per year containing security fixes and critical bug fixes. To build Xen Project software from source, you can either download a source release or you can fetch the source tree from the source repository. Each source release and the source tree contain a README file in the root directory, with detailed build instructions for the hypervisor. The release notes for each release also contain build instructions and so does the Compiling Xen Project software page.


Xen Project software in Linux/Unix Distributions Most Linux and many Unix distributions contain built binaries of the Xen Project hypervisor that can be downloaded and installed through the native package management system. If your Linux/Unix distribution includes the hypervisor and a Xen Project-enabled kernel, we recommend to use them as you will benefit from ease of install, good integration with the distribution, support from the distribution, provision of security updates etc. Installing the hypervisor in a distribution typically requires the following basic steps: a) Install your favourite distribution, b) Install Xen Project package(s) or meta-package, c) check boot settings and d) reboot. After the reboot, your system will run your favourite Linux/Unix distribution as Control Domain on top of the hypervisor.

Host and Guest Install

The following documents

This table contains a list of Xen Project resources for various Linux and Unix distributions.

[edit]

Distro Main website Description Resources
Arch Linux archlinux.org Arch Linux is a lightweight and flexible Linux® distribution that tries to “keep it simple”.


Alpine Linux alpinelinux.org A security-oriented, lightweight Linux distribution based on musl libc and busybox.


CentOS 5 centos.org CentOS is an Enterprise-class Linux Distribution derived from sources freely provided to the public by a prominent North American Enterprise Linux vendor. CentOS conforms fully with the upstream vendor's redistribution policy and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork.) CentOS is free.


CentOS 6 centos.org CentOS is an Enterprise-class Linux Distribution derived from sources freely provided to the public by a prominent North American Enterprise Linux vendor. CentOS conforms fully with the upstream vendor's redistribution policy and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork.) CentOS is free.

CentOS 6.0 - 6.3 does not include Xen Project software, but you can get support from various sources. The following articles may be useful

CentOS 6.4+ does include Xen Project support and can be used as a dom0 and domU out-of-the-box, thanks to the Xen4CentOS project

Xen packages in CentOS 6 and commercial support are also available from "Xen made easy!"


Debian debian.org The Debian project produces an entirely free operating system that empowers its users to be in control of the software running their computers.


Fedora fedoraproject.org Fedora is a RPM-based distribution with a 6-month release cycle, and is the community-supported base of RHEL releases.


FreeBSD freebsd.org FreeBSD® is an advanced operating system for modern server, desktop, and embedded computer platforms.


Finnix finnix.org Finnix is a sysadmin utility Linux LiveCD, and includes out-of-the-box Xen Project guest support.


Gentoo Linux gentoo.org Gentoo Linux is a special flavor of Linux that can be automatically optimized and customized for just about any application or need. Extreme performance, configurability and a top-notch user and developer community are all hallmarks of the Gentoo experience.


NetBSD netbsd.org NetBSD is a free, fast, secure, and highly portable Unix-like open source operating system.


Oracle Linux oracle.com Oracle Corporation distributes Oracle Linux the Unbreakable Enterprise Kernel. Oracle states that the Unbreakable Enterprise Kernel is compatible with RHEL, Oracle middleware and 3rd-party RHEL-certified applications. Oracle Linux supports KVM, Xen Project, and Oracle VM Server for x86, which is based on Xen.


openSuSE opensuse.org openSuSE is a free and Linux-based operating system for your PC, Laptop or Server.


Red Hat Enterprise Linux (RHEL) 5.x redhat.com RHEL 5.x includes the Xen Project 3.4 Hypervisor as well as a Xen Project-enabled kernel, and can be used as a dom0 and domU


Red Hat Enterprise Linux (RHEL) 6.x redhat.com RHEL 6.x does not include the Xen Project Hypervisor. But, a Dom0 capable kernel, Xen Project hypervisor, and libvirt packages for use with RedHat Enterprise Linux 6 and its derivatives are available from either the Xen4CentOS project or the "Xen made easy!" effort.


Ubuntu ubuntu.com Fast, secure and stylishly simple, the Ubuntu operating system is used by 20 million people worldwide every day.



Getting Help!

The Xen Project community contains many helpful and friendly people. We are here for you. There are several ways to get help and keep on top of what is going on!

  • Read News!
  • Read Documentation!
  • Contact other users, to ask the questions and discuss the hypervisor or other Xen Project-related projects

News Sources

Documentation

Documentation for projects hosted on XenProject.org is available on the Xen Project Wiki. Our wiki is active and community maintained. It contains a lot of useful information and uses categories extensively to make it easy to find information. You may also want to check:

Mailing Lists

Search Mailing Lists All XenProject.org mailing lists are archived using the MarkMail system at xen.markmail.org. Before you ask a question, it is worth checking whether somebody else has asked the question before

Main Mailing Lists XenProject.org maintains a number of mailing lists for users of the hypervisor and other projects. English is used by readers on this list.

  • xen-users is the list for technical support and discussions for the Xen Project hypervisor. If you are not sure where your question belongs start here!
  • xen-api is the DEPRECATED list for technical support and discussions for the Xen Cloud Platform (XCP).


IRC

Internet Relay Chat (IRC) is a great way to connect with Xen Project community members in real time chat and for support.

  • ##xen is the channel for technical support and discussions for the Xen Project hypervisor. If you are not sure where your question belongs start here!
  • Check out our IRC page if you are not familiar with IRC.

Other places

There are a number of other places, where you can get help on Xen Project software. For example:

Raising Bugs

If you find a bug, you can report bugs against the software. Before you raise a bug, please read Reporting Bugs!

Roadmaps, Release Cadence, Maintenance Releases

The Xen Project community releases the Xen Project Hypervisor with a release cadence of 6 months (in June and December of each year). Roadmap information is tracked at Xen Roadmap. You can find information on the maintenance release cycle at Xen Project Maintenance Releases.

Also See

Installation

Release Information

Specialist Topics: Networking, Performance, Security, NUMA, VGA, ...

Specialized Xen Project topics:

  • Category:Networking contains articles related to networking
  • Category:NUMA contains all articles related to the running (or to improving the support for doing so) of the Xen Project Hypervisor on NUMA architectures
  • Category:Performance contains documents, tuning instructions and benchmarks related to the performance of Xen Project software
  • Category:Security contains documents related to Xen Project security
  • Category:VGA contains documents related to VGA, VTd, GPY passthrough, etc.

FAQs, HowTos, ...

Languages Language: English  • Deutsch • español • français • 日本語 • 한국어 • português do Brasil • русский • 中文
Retrieved from "https://wiki.xenproject.org/index.php?title=Xen_Project_Software_Overview&oldid=18362"