Memory hot-unplug with Qemu and libvirt

Removing of memory with Qemu is supported since version 2.4 and allows to remove hot plugged memory as described previously.

Requirements

For Linux to support hot-remove of memory the options CONFIG_MEMORY_HOTREMOVE and CONFIG_MIGRATION have to be enabled.

In the following steps I assume the following config:

<domain type="kvm">
...
<devices>
...
<memory model='dimm'>
<target>
<size unit='KiB'>131072</size>
<node>0</node>
</target>
</memory>
<memory model='dimm'>
<target>
<size unit='KiB'>262144</size>
<node>0</node>
</target>
</memory>
</devices>
</domain>

virsh

As with memory hotplug we need a xml file to declare the device which should be removed:

<memory model='dimm'>
<target>
<size unit='MiB'>128</size>
<node>0</node>
</target>
</memory>

And the command to remove memory:

virsh detach-device <vm> <xml file> --config --live

If the requirements are meet, the guest has now 128MB less memory. The xml file has to look the same as the one that was used to add memory. If you try this command again, you get the following error message:

error: Failed to detach device from <xml file>
error: operation failed: matching memory device was not found

Hot-remove with python and libvirt API

For Debian based systems python-libvirt or python3-libvirt is needed. On openSuse the package is named libvirt-python. The following snippet would do everything which the previous virsh command did for us.

import libvirt
conn = libvirt.open()
vm = conn.lookupByName("vm_name")
xml = "<memory model='dimm'><target><size unit='MiB'>128</size><node>0</node></target></memory>"
vm.detachDeviceFlags(xml,
libvirt.VIR_DOMAIN_AFFECT_LIVE|libvirt.VIR_DOMAIN_AFFECT_CONFIG)

Possible Problems

libvirt reports success but memory is not decreased

This can happen if the guest kernel does not support hot-remove of memory. I would expect an error message from Qemu or libvirt but I haven’t looked into it yet

Only some modules go offline

This can happen if some modules contain non migratable memory. Currently only anonymous pages and page caches can be migrated. This can be avoided at hotplug with online_movable instead of just online.