The Linux Process Journey — migration (kernel thread)
One of the goals of an operating system is to handle and balance resources across the hardware of the compute. In order to do that, Linux has a kernel thread named “migration” which has an instance on every vCPU. By the way, the naming format is “migration/N” where N is the id of the vCPU.
By default threads are not constrained to a vCPU and can be migrated between them in the next call to “schedule()” (which call the main scheduler function, which is “__scheduler()” — https://elixir.bootlin.com/linux/latest/source/kernel/sched/core.c#L6544). It is done mainly in case the scheduler identifies an unbalanced across the runqueues (the queue in which processes which are in ready/runnable state are waiting to use the processor) of the vCPUs.
It is important to state that we can influence this flow by setting the affinity of a thread (for more read “man 2 sched_setaffinity”. We will talk about that in a future post). There could be performance, cache and other impacts for doing that (but that is also a topic for a different writeup).
In summary, what the kernel thread “migration” does is to move threads from highly loaded vCPUs to others which are less crowded (by inserting them to a different run-queue). A function which is used by “migration” in order to move a task to a new run-queue is “move_queued_task” (https://elixir.bootlin.com/linux/latest/source/kernel/sched/core.c#L2325).
I have created a small demo which shows the working of “migration”. For that I have created a VM running Ubuntu 22.04 with 3 vCPUs. In order to trace the usage of “move_queue_task” I have used bpftrace with the following command: sudo bpftrace -e ‘kfunc:move_queued_task { printf(“%s moved %s to %d CPU\n”,curtask->comm,args->p->comm,args->new_cpu); }’. The output of the command is shown below. The one-liner prints: the name of the task calling “move_queued_task”, the name of the task which is moved and id the vCPU which the task is moved to.
You can also check my twitter account — @boutnaru (https://twitter.com/boutnaru)
See you next time ;-)
