Paravirt Linux CPU Hotplug

Why does not xl vcpu-set work as expected when I try to increase the number of VCPUs?

First, you need maxvcups= to be high enough in the guest configuration settings. You can then check the online vs offline vcpus for a guest by doing xl vcpu-list GUEST.

Second, you are probably hitting the other limitation, which requires an operation to be performed within the guest. It is likely that hotplug events are not handled correctly by udevdin the target domain. To identify the problem, run udevadm monitor as root in target domain, then plug CPUs by doing xl vcpu-set GUEST NUMBER. If you can see similar output:

KERNEL[1327937839.577165] add      /devices/system/cpu/cpu1 (cpu)
UDEV  [1327937839.577368] add      /devices/system/cpu/cpu1 (cpu)
KERNEL[1327937839.577581] add      /devices/system/cpu/cpu2 (cpu)
UDEV  [1327937839.577684] add      /devices/system/cpu/cpu2 (cpu)

then CPUs are correctly added to system and udev can see it. You then have to write to the corresponding sysfs nodes to bring those CPUs online.

To enable the hot-plugged CPUs manually,

       echo 1 > /sys/devices/system/cpu/cpu15/online

And to automate the process,

       ls -lhF /etc/udev/rules.d/
       ls -lhF /lib/udev/rules.d/
       cat > /etc/udev/rules.d/cpu-online.rules <<-EOF
       SUBSYSTEM=="cpu", ACTION=="add", RUN+="/bin/sh -c 'echo 1 > %S%p/online'"
       EOF

then tell udevd to reload its rules with udevadm control --reload-rules (or restart udevd).

Watching the system logs (tail -F /var/log/messages) helps, as it shows the output of potentially failing udev rules and eventually some message related to a new enabled CPU.

If you've checked every options and configurations and believe that the problem is caused by a bug, please follow instructions in Reporting_Bugs_against_Xen to submit bug report.

Additional Notes

The following RUN rule may be used as well (checking for the online file beforehand),

RUN+="/bin/sh -c '[ ! -e %S%p/online ] || echo 1 > %S%p/online'"

Background

- See this mailing list thread for discussion of the rationale for this behavior. - https://www.linuxquestions.org/questions/slackware-14/slackware-udev-rules-4175465771/ - https://lwn.net/Articles/528976/ - https://www.kernel.org/doc/html/v4.18/core-api/cpu_hotplug.html - https://www.kernel.org/doc/Documentation/power/suspend-and-cpuhotplug.txt