Paravirt Linux CPU Hotplug: Difference between revisions
No edit summary |
(udev rules: $devpath --> %S%p) |
||
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
Why does not xl vcpu-set work as expected when I try to increase the number of VCPUs? |
=== Why does not xl vcpu-set work as expected when I try to increase the number of VCPUs? === |
||
First, you need <code>maxvcups=</code> to be high enough in the guest configuration settings. You can then check the online vs offline vcpus for a guest by doing <code>xl vcpu-list GUEST</code>. |
|||
Most likely that udev event is not handled correctly in the target domain. |
|||
Second, you are probably hitting the other limitation, which requires an operation to be performed within the guest. It is likely that hotplug events are not handled correctly by <code>udevd</code>in the target domain. To identify the problem, run <code>udevadm monitor</code> as root in target domain, then plug CPUs by doing <code>xl vcpu-set GUEST NUMBER</code>. If you can see similar output: |
|||
KERNEL[1327937839.577165] add /devices/system/cpu/cpu1 (cpu) |
KERNEL[1327937839.577165] add /devices/system/cpu/cpu1 (cpu) |
||
| Line 10: | Line 10: | ||
UDEV [1327937839.577684] add /devices/system/cpu/cpu2 (cpu) |
UDEV [1327937839.577684] add /devices/system/cpu/cpu2 (cpu) |
||
then CPUs are correctly added to system. |
then CPUs are correctly added to system and <code>udev</code> can see it. You then have to write to the corresponding sysfs nodes to bring those CPUs online. |
||
To enable the hot-plugged CPUs manually, |
|||
All you need to do now is write to corresponding sysfs nodes to bring those CPUs online. |
|||
echo 1 > /sys/devices/system/cpu/cpu15/online |
|||
To automate this process, simply create a `cpu-online.rules` under /etc/udev/rules.d in target domain and restart udevd. |
|||
And to automate the process, |
|||
Content of `cpu-online.rules`: |
|||
ls -lhF /etc/udev/rules.d/ |
|||
| ⚫ | |||
ls -lhF /lib/udev/rules.d/ |
|||
cat > /etc/udev/rules.d/cpu-online.rules <<-EOF |
|||
| ⚫ | |||
EOF |
|||
then tell <code>udevd</code> to reload its rules with <code>udevadm control --reload-rules</code> (or restart <code>udevd</code>). |
|||
If you cannot see hotplug events, probably you're hitting other limitations such as max VCPU quotation, do check against Xen configuration documents. |
|||
Watching the system logs (tail -F /var/log/messages) helps, as it shows the output of potentially failing udev rules and eventually some message related to a new enabled CPU. |
|||
| ⚫ | |||
| ⚫ | |||
=== Additional Notes === |
|||
The following RUN rule may be used as well (checking for the online file beforehand), |
|||
RUN+="/bin/sh -c '[ ! -e %S%p/online ] || echo 1 > %S%p/online'" |
|||
=== Background === |
|||
- See this [http://lists.xen.org/archives/html/xen-devel/2010-05/msg00516.html mailing list thread] for discussion of the rationale for this behavior. |
|||
- https://www.linuxquestions.org/questions/slackware-14/slackware-udev-rules-4175465771/ |
|||
- https://lwn.net/Articles/528976/ |
|||
- https://www.kernel.org/doc/html/v4.18/core-api/cpu_hotplug.html |
|||
- https://www.kernel.org/doc/Documentation/power/suspend-and-cpuhotplug.txt |
|||
[[Category:Users]] |
[[Category:Users]] |
||
Latest revision as of 14:22, 12 November 2018
Why does not xl vcpu-set work as expected when I try to increase the number of VCPUs?
First, you need maxvcups= to be high enough in the guest configuration settings. You can then check the online vs offline vcpus for a guest by doing xl vcpu-list GUEST.
Second, you are probably hitting the other limitation, which requires an operation to be performed within the guest. It is likely that hotplug events are not handled correctly by udevdin the target domain. To identify the problem, run udevadm monitor as root in target domain, then plug CPUs by doing xl vcpu-set GUEST NUMBER. If you can see similar output:
KERNEL[1327937839.577165] add /devices/system/cpu/cpu1 (cpu) UDEV [1327937839.577368] add /devices/system/cpu/cpu1 (cpu) KERNEL[1327937839.577581] add /devices/system/cpu/cpu2 (cpu) UDEV [1327937839.577684] add /devices/system/cpu/cpu2 (cpu)
then CPUs are correctly added to system and udev can see it. You then have to write to the corresponding sysfs nodes to bring those CPUs online.
To enable the hot-plugged CPUs manually,
echo 1 > /sys/devices/system/cpu/cpu15/online
And to automate the process,
ls -lhF /etc/udev/rules.d/
ls -lhF /lib/udev/rules.d/
cat > /etc/udev/rules.d/cpu-online.rules <<-EOF
SUBSYSTEM=="cpu", ACTION=="add", RUN+="/bin/sh -c 'echo 1 > %S%p/online'"
EOF
then tell udevd to reload its rules with udevadm control --reload-rules (or restart udevd).
Watching the system logs (tail -F /var/log/messages) helps, as it shows the output of potentially failing udev rules and eventually some message related to a new enabled CPU.
If you've checked every options and configurations and believe that the problem is caused by a bug, please follow instructions in Reporting_Bugs_against_Xen to submit bug report.
Additional Notes
The following RUN rule may be used as well (checking for the online file beforehand),
RUN+="/bin/sh -c '[ ! -e %S%p/online ] || echo 1 > %S%p/online'"
Background
- See this mailing list thread for discussion of the rationale for this behavior. - https://www.linuxquestions.org/questions/slackware-14/slackware-udev-rules-4175465771/ - https://lwn.net/Articles/528976/ - https://www.kernel.org/doc/html/v4.18/core-api/cpu_hotplug.html - https://www.kernel.org/doc/Documentation/power/suspend-and-cpuhotplug.txt
