Archived/XenTest/FunctionalTests

Functional Test Matrix

---

Linux Compliance and Conformance Tests

Test	Description	Upstream	Parameters	Criteria	Variables	Votes	Status
LSB 3	Linux Standards Base	http://www.freestandards.org/en/TestGroup http://wiki.debian.org/LsbRuntimeCompliance	full test (exclude desktop?)	native == dom0 results, dom0 == domU results	Linux distributions in guests, PCPUs, VCPUs, RAM, backing store, file systems, device domains	3	proposed
LTP 20060306	Linux Test Project	http://www.linuxtestproject.org/	|./runalltests.sh -l log -o results -t 60m	native == dom0 results, dom0 == domU results	Linux distributions in guests, PCPUs, VCPUs, RAM, backing store, file systems, device domains	5	proposed
Open POSIX Test Suite	test for POSIX compliance	http://posixtest.sourceforge.net/	|make build-tests; make run-tests; make functional-tests; make stress-tests	native == dom0 results, dom0 == domU results	Linux distributions in guests, PCPUs, VCPUs, RAM, backing store, file systems, device domains	1	proposed

Domain Tests

Test	Description	Upstream	Parameters	Criteria	Variables	Votes	Status
Create	can a guest domain be created? (is there an xm-test for this?)		|xm create domU	guest starts, shows as present in xm list	PCPUs, VCPUs, PV and HVM guests, backing store	5	to be written (?)
Destroy	can a guest domain be destroyed (not halted)? (is there an xm-test for this?)		|xm destroy domU	guest stops "immediately"	PCPUs, VCPUs, PV and HVM guests, backing store	5	to be written (?)
Migration	can a running domain be moved on from/to localhost? a remote host? (is there an xm-test for this?)			domain moves without disruption of processing	PV and HVM guests, backing store	4	to be written (?)
Mixed DomUs	can multiple, non-homogeneous guests run simultaneously?				number of guests, PV vs HVM, PCPUs, VCPUs, which guests, scheduler, backing store, file systems, device domains	5	to be written
Paused Migration	can a paused domain be moved from/to localhost? a remote host? (is there an xm-test for this?)			domain moves without disruption of processing	PV and HVM guests, backing store	3	to be written (?)
Paravirtualized Drivers for Fully Virtualized Domains						2	to be written (?)
Reboot	can a guest domain be rebooted? (is there an xm-test for this?)		|xm reboot domU	guest reboots, returns to login prompt, backing store uncorrupted	PCPUs, VCPUs, PV and HVM guests, backing store	5	to be written (?)
Save/Restore	can a running domain be saved/restored on localhost? a remote host? (is there an xm-test for this?)			domain stops/restarts without disruption of processing	PV and HVM guests, backing store	4	to be written (?)
Start	can a guest domain be started? NB: in 3.0.4 this will be different than xm create but is not currently; is there an xm-test for this?		|xm start domU -c	guest boots and presents login prompt	PCPUs, VCPUs, RAM, PV and HVM guests, backing store	5	to be written (?)
Shutdown	can a guest domain be shutdown? (is there an xm-test for this?)		|xm shutdown domU	guest stops, no longer present in xm list, backing store uncorrupted	PCPUs, VCPUs, PV and HVM guests, backing store	5	to be written (?)
Validate Separation	demonstrate that guests cannot interfere with one another				PCPUs, VCPUs, PV and HVM guests, backing store	3	to be written

Tool Tests

Test	Description	Upstream	Parameters	Criteria	Variables	Votes	Status
kexec/kdump	tests operation of kexec and kdump	Xen		native == dom0 results, dom0 == domU results	Linux distributions in guests, PCPUs, VCPUs, RAM, backing store, file systems, device domains	3	to be written
xentop operation	does xentop produce correct results?	Xen		when all parameters are invoked, are the results reasonable?	CPU load, number of domUs running	2	to be written?
xm-test Test Suite	tests operation of xm command	Xen	run full suite	native and domU should not work at all, dom0 should pass all tests	Linux distributions in guests, PCPUs, VCPUs, RAM, backing store, file systems, device domains	5	proposed

General Tests

Test	Description	Upstream	Parameters	Criteria	Variables	Votes	Status
Distro Installation Test	Can a distro install Xen and a dom0 that boots?		default install	installation succeeds, system boots into dom0	distro, file system, PCPUs, VCPUs, driver domains	5	proposed
Hotplug Tests	does hotplug operate properly?		dom0, agreed upon Linux distros, Windows, others?	native results == dom0 results, dom0 == domU	dom0, domU, distro, backing store, PCPUs, VCPUs, paravirtual drivers	2	to be written (?)
Linux Kernel Compile (2.6.16 source)	Compile the Linux kernel; provides broad brush pass/fail on whether Linux is behaving		default config file	kernel compiles without error in any domain	dom0, domU, distro, backing store, PCPUs, VCPUs, driver domains	5	proposed
MCA Tests	does MCA (or its equivalent) operate properly?		dom0, agreed upon Linux distros, Windows, others?	native results == dom0 results, dom0 == domU	dom0, domU, distro, backing store, PCPUs, VCPUs, paravirtual drivers	3	to be written (?)
Ping	Is the bridge behaving?		dom0 with minimum of two domUs running	dom0 can ping each domU, each domU can ping dom0, domUs can ping each other	dom0, domU, distro, backing store, PCPUs, VCPUs, driver domains	3	to be written
Threading Tests	do threads operate properly?		dom0, agreed upon Linux distros, Windows, others?	native results == dom0 results, dom0 == domU	dom0, domU, distro, backing store, PCPUs, VCPUs, paravirtual drivers	3	to be written (?)
VT Testing	do VTi, AMD-V run HVMs properly?		dom0, agreed upon Linux distros, Windows, others?	each HVM can boot, a user can login and performa basic tasks	dom0, domU, distro, backing store, PCPUs, VCPUs, paravirtual drivers	5	to be written
X11 Testing	does X Windows run in dom0?		run startx	basic X Windows desktop starts and functions normally	dom0, PCPUs, VCPUs, file system	2	to be written

---

Links

• XenTest/KeyToColumns
• XenTest
• XenTest/RegressionTests
• XenTest/PerformanceTests
• XenTest/StressTests