Cooking with Azure
Published in

Cooking with Azure

Using KubeVirt in Azure Kubernetes Service — part 1: Introduction

A primer on using kubevirt to run VirtualMachines as container workloads on the Azure Kubernetes service. This first post is intended as an introduction to KubeVirt. You can read the second part here.

KubeVirt is an open source project whose aim is to help running virtual machine workloads natively within a Kubernetes cluster, by extending the core with components that facilitate the process (using Custom Resource Definitions like VirtualMachines and Disks). It allows for co-existence of mixed workloads and smoother transition from legacy VMs to containers (like a step-wise microservices adoption) unified by the control plane and the familiar tooling (kubectl and the optional virtctl) of Kubernetes.

Why kubevirt?

For a recent project where we needed to run VMs side-by-side containers, we evaluated a number of options to run VM’s side-by-side with containers:

- Azure Service Operator (ASO)

The Azure Service Operator is a new project born within Commercial Software Engineering aimed at managing Azure resources as native Kubernetes objects. Altough very promising, we put it aside as we recognize it’s still rough around the edges (see this [issue](https://github.com/Azure/azure-service-operator/issues/1167)).

- Terraform Operator for Kubernetes

A new project from Hashicorp that aims at allowing generic Terraform templates lifecycle management natively in Kubernetes. It could have been a good alternative (after all, we did develop extensive templates for VM/VMSS in the first project phase) but it’s still considered alpha so we will not research it further.

Both the above projects are interesting and we’ll be considered in the future but we’re focusing on kubevirt to spin up the necessary VMs for the time being.

KubeVirt Architecture

KubeVirt runs VirtualMachine as embedded in pods running an hypervisor (commonly, `libvirtd`) and exposes them via Kubernetes services:

Kubevirt is composed of several components:

- virt-controller (cluster component, watches for `VirtualMachine` CRDs)

- virt-api (cluster component, the interface to the controller)

- virt-handler (DaemonSet)

- virt-launcher (for every VM pod)

Implementation

Kube Virt (KV from now on) aims to maintain the so-called Application Binary Interface (ABI) stability (a fancy way to say that the aim is to have the same UX as using native VMs) and management capabilities for VMs in Kubernetes. Kube Virt adds a basic CRD to the Kubernetes API called `VirtualMachine` to abstract the concept of a VM; the controller watches for this kind of object being added/modified/deleted and when one is created with `spec.running` set to true it instantiates a new object of kind `VirtualMachineInstance` which represents the actual VM.

Importantly, the `VirtualMachineInstance` can exposed as a regular Kubernetes service of type `LoadBalancer`, `NodePort` or `ClusterIP`. It uses the standard label/selector pair to attach the service to the `VirtualMachineInstance`.

VirtualMachine Object

The VM object defines the specs for the VirtualMachine and contains the spec for the VirtualMachineInstance (under spec.domain):

Notable in this implementation:

- `spec.running` indicates if the VM should be running or stopped. Can be manipulated by `kubectl patch` or manual editing or directly with `virtctl`:

$ virtctl start myvm
#is equvalent to
kubectl patch virtualmachine myvm --type merge -p '{"spec":{"running":false}}'

- `resources.requests` have the same effects as on containers, allocating resources to the virtual machines.

- `disks` in the VM are mapped to regular Kubernetes persisten `volumes` of type `containerDisk`, which are prepared beforehand using the Containerized Data Importer. The disk can also come directly from an URL using the `dataVolumeTemplates` (see Part 2).

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alessandro Vozza

Alessandro Vozza

Full time Cloud Pirate, software engineer at Microsoft, public speaker, community organiser and mentor. Opinions are mine’s, facts are facts.