NUMA node affinity in the Xen hypervisor: Difference between revisions
(Created page with "= Setting a Domain’s node affinity = == Hypercall: <code>XEN_DOMCTL_setnodeaffinity</code> == It can be called from <code>libxenctrl</code> and <code>libxl</code> === <code>libxenctrl</code>: <code>xc_domain_node_setaffinity()</code> === This call requires the <code>domid</code> of the created domain and an <code>xc_nodemap_t</code>: <source lang="c">int xc_domain_node_setaffinity(xc_interface *xch, uint32_t domid,...") |
mNo edit summary |
||
Line 9: | Line 9: | ||
This call requires the <code>domid</code> of the created domain and an <code>xc_nodemap_t</code>: |
This call requires the <code>domid</code> of the created domain and an <code>xc_nodemap_t</code>: |
||
<syntaxhighlight lang="c"> |
|||
int xc_domain_node_setaffinity(xc_interface *xch, |
|||
uint32_t domid, |
uint32_t domid, |
||
xc_nodemap_t nodemap) |
xc_nodemap_t nodemap) |
||
</syntaxhighlight> |
|||
Source: https://github.com/xen-project/xen/blob/master/tools/libs/ctrl/xc_domain.c#L122 |
Source: https://github.com/xen-project/xen/blob/master/tools/libs/ctrl/xc_domain.c#L122 |
||
Line 36: | Line 38: | ||
While this call cannot influence the past, whereas domain_create() already created the domain and allocated the internal Xen data structures, the setting node_affinity early can change the course of memory allocations for the domain. |
While this call cannot influence the past, whereas domain_create() already created the domain and allocated the internal Xen data structures, the setting node_affinity early can change the course of memory allocations for the domain. |
||
= Used by |
= Used by = |
||
* [[Improved NUMA memory allocation]] |
|||
** [[NUMA node-specifc memory allocation]] |
|||
** [[Walkthrough: VM build using xenguest]] |
|||
[[Category:NUMA]] |
[[Category:NUMA]] |
Revision as of 10:14, 11 February 2025
Setting a Domain’s node affinity
Hypercall: XEN_DOMCTL_setnodeaffinity
It can be called from libxenctrl
and libxl
libxenctrl
: xc_domain_node_setaffinity()
This call requires the domid
of the created domain and an xc_nodemap_t
:
int xc_domain_node_setaffinity(xc_interface *xch,
uint32_t domid,
xc_nodemap_t nodemap)
Source: https://github.com/xen-project/xen/blob/master/tools/libs/ctrl/xc_domain.c#L122
Xen Hypervisor implementation
For setnodeaffinity
, do_domctl()
calls domain_set_node_affinity()
The entry point for al DOMCTL
phypercalls is the function do_domctl()
:
When asked for XEN_DOMCTL_setnodeaffinity, it calls xenctl_bitmap_to_nodemask()
to convert &op->u.nodeaffinity.nodemap
to nodemask_t
and then calls domain_set_node_affinity(d, &new_affinity);
with the nodemask: https://github.com/xen-project/xen/blob/master/xen/common/domctl.c#L516
domain_set_node_affinity()
Starts at https://github.com/xen-project/xen/blob/master/xen/common/domain.c#L943
If the new nodemap does not intersect the node_online_map
, it returns -EINVAL
.
On success, it disables the automatic affinity feature of Xen, where Xen tries to deduce the affinity of the domain from the affinty of its vCPUs for this domain and updates the domain’s node_affinity
for memory allocations using the buddy allocator.
It also calls into the Xen scheduler to notify it of the change. If the Xen scheduler does not have any vCPUs at that moment, the Xen scheduler notification does nothing.
However, setting an explicit node_affinity for the domain changes the NUMA nodes to allocate from before the vCPU affinity is set, and it could alter the behavior of the scheduler in theory as well.
While this call cannot influence the past, whereas domain_create() already created the domain and allocated the internal Xen data structures, the setting node_affinity early can change the course of memory allocations for the domain.