Update Intel microcode on CentOS using Xen

Image for post
Image for post
Meltdown, Spectre and MDS vulnerabilities

With all the recent vulnerabilities on Intel CPUs, it’s more important than ever to upgrade CPU’s firmware to mitigate critical vulnerabilities such as Meltdown, Spectre, Speculative Store Bypass or MDS. Instead of updating the system bios and risk instability of a server in production, most Linux distributions provide microcode_ctl package to update the CPU microcode when booting. There’s a few more steps to do on CentOS with Xen to apply the microcode and this is frequently asked on IRC as well as in mailing lists so I will cover it here.

It’s not possible to directly load the microcode file provided by CentOS 6. We need to extract it to a blob file using iucode_tool:

iucode_tool -w /boot/microcode.blob /lib/firmware/microcode.dat

Add “ucode=-1” to the Xen command line in /boot/grub/grub.conf and specify the microcode file located in /boot directory, which is named microcode.blob in this example, then reboot:

kernel /xen.gz dom0_mem=1792M,max:2048M dom0_max_vcpus=4 cpuinfo com1=115200,8n1 console=com1,vga xpti=no-dom0 crashkernel=512M@64M loglvl=all guest_loglvl=all ucode=-1
module /vmlinuz-4.9.184–35.el6.x86_64 ro root=UUID=d1358533-f9cb-4866–80b0–60b7a33472fe rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM elevator=deadline nohz=off max_loop=128 crashkernel=auto
module /initramfs-4.9.184–35.el6.x86_64.img
module /microcode.blob

It’s pretty simple to load microcode with CentOS 7 using the microcode_ctl package. We only need the following fix because RedHat now refuses to apply the update inside of a Xen dom0:

mkdir -p /etc/microcode_ctl/ucode_with_caveats
touch /etc/microcode_ctl/ucode_with_caveats/force

Rebuild initramfs (only once) to include the microcode after doing the fix:

yum reinstall microcode_ctl

Add “ucode=scan” to the default Xen command line in /etc/default/grub:

GRUB_CMDLINE_XEN_DEFAULT=”dom0_mem=1792M,max:2048M dom0_max_vcpus=4 ucode=scan cpuinfo com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl=all xpti=no-dom0"

Rebuild grub config:

grub2-mkconfig -o /boot/grub2/grub.cfg

Ensure ucode parameter is in grub2 configuration then reboot:

multiboot /xen-4.8.5.21.g752fb21a29–1.el7.gz placeholder dom0_mem=1792M,max:2048M dom0_max_vcpus=4 ucode=scan cpuinfo com1=115200,8n1 console=com1,vga loglvl=all guest_loglvl=all xpti=no-dom0 ${xen_rm_opts}
module /vmlinuz-4.9.184–35.el7.x86_64 placeholder root=UUID=52b0229a-e12f-47f6-b926–0210e4c7fd8f ro crashkernel=auto elevator=deadline nohz=off max_loop=128 console=hvc0 console=tty0 earlyprintk=xen nomodeset
module — nounzip /initramfs-4.9.184–35.el7.x86_64.img

Instead of using the microcode provided by the OS microcode_ctl package, it’s possible to use the latest one from Intel as they are available in a GitHub repository. Utility cpuid can be used to find the right Intel microcode. In this case, E5–2670 v2 has the following microcode: 06–3E-04

family = Intel Pentium Pro/II/III/Celeron/Core/Core 2/Atom, AMD Athlon/Duron, Cyrix M2, VIA C3 (6)
model = 0xe (14)
stepping id = 0x4 (4)
extended family = 0x0 (0)
extended model = 0x3 (3)

Once we know the right microcode, we can download it in /boot, use “ucode=-1” and add a new module line, same as the previous CentOS 6 example, with the microcode blob name:

module /06–3E-04

Xen dmesg will report if the microcode has been updated for all CPUs from the current one in the bios to the latest one available in the microcode blob:

[root@node2 ~]# xl dmesg|grep -i microcode
(XEN) microcode: CPU0 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU2 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU4 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU6 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU8 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU10 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU12 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU14 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU16 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU18 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU20 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU22 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU24 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU26 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU28 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU30 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU32 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU34 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU36 updated from revision 0x42d to 0x42e, date = 2019-03-14
(XEN) microcode: CPU38 updated from revision 0x42d to 0x42e, date = 2019-03-14
[root@node2 ~]#

We can also confirm the new microcode in the virtual guests (domU) however they have to be rebooted if they have been paused instead of shut down:

[root@monitoring ~]# grep microcode /proc/cpuinfo
microcode : 0x42e
microcode : 0x42e
microcode : 0x42e
microcode : 0x42e
[root@monitoring ~]#

The following GitHub repo provide a pretty complete script to test your system against those recent vulnerabilities: https://github.com/speed47/spectre-meltdown-checker

Image for post
Image for post

If the microcode has been applied correctly and it’s recent enough to include latest patch from Intel, all CVE should be green as above. The Linux kernel as well as Xen also have to be updated because those vulnerabilities are mitigated by a combination of software and firmware. Note that Intel hasn’t patched all CPUs so if it’s an old one, it’s possible that the microcode is too old to secure against some those CVEs. See the link below to find if your CPU is still supported by Intel.

Written by

I'm an IT architect involved in the hosting business. I spend my free time doing R&D, security, packaging rpms and work on startups!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store