Intelpocalypse: goodbye fast system calls

Intel announces one security vulnerability after another making system calls slower and slower. In May 14 they announced MDS (aka ZombieLoad) vulnerability. Earlier, in 2018, there was announced Metdown.

Modern Linux kernel is compiled with Kernel page table isolation (KPTI) to prevent Metldown. Essentially, KPTI is just a removal of old technique to optimize system calls, aka lazy TLB: kernel space is mapped to all page tables for user space processes, so there is no need to flush 1 layer caches on kernel/user-space context switches. Performance impact is serious: up to 20% for Nginx (MariaDB got even 40% for certain workloads).

The numbers are for systems with PCID (the feature is available for all modern processors in recent years). Without PCID you may hit up to 90% performance impact. PCID introduces tagging for TLB, so that you can keep some cached page mappings and don’t need to invalidate L1 TLB cache twice on each system call.

MDS goes further in slowing down system calls, it introduces mds_clear_cpu_buffers() called on each context switch. Performance impact seems not so huge as for the Meltdown prevention, but it’s clear that system calls become even more slow.

The good news is that Tempesta FW works in kernel space, so there is no context switches and KPTI and MDS do not affect our performance at all. Moreover, we accurately program our most performance crucial code (HTTP processing) in assembly and use retpoline Spectre prevention only where it’s necessary. Retpoline may have up to 15% performance impact, but, fortunately, not each indirect jump must use retpoline to be safe against Spectre.