Kata Containers testing and packaging powered by the cloud

For any project, testing is the key to stability. Since Kata Containers uses hardware virtualization to provide an extra layer of isolation for containers, our testing requirements are more complicated: each node needs to support running virtual machines (VMs). This is accomplished through either nested-virtualization support on virtual machines, or virtualization directly on bare-metal.

The Kata project decided to run continuous integration (CI) in the cloud; maintaining our own infrastructure was not a viable option for our open and distributed development team. In addition, we wanted to make sure that Kata Containers runs properly on different clouds, so it made sense to run our CI using the services of multiple cloud service providers (CSPs).

The current cloud providers that have generously donated resources to the Kata project are:

• Amazon Web Services* (AWS)
• Google* Compute Engine
• IBM*
• Microsoft Azure*
• openSUSE Open Build Service*
• PackageCloud*
• Packet*
• Vexxhost*

We use each of these resources to varying degrees for our CI and testing efforts.

AWS:

With the introduction of Firecracker* hypervisor support in Kata Containers, baremetal verification of Kata + Firecracker on AWS became a priority for the Kata project. Thanks to their generous donation, we are now in the process of migrating our Firecracker jobs to AWS.

Google:

On Google Compute Engine (GCE), we have run basic verification of our GCP* install directions. We also have it configured for running tests on PR changes and have tuned our CI usage to take advantage of our resource allocation.

IBM:

IBM has sponsored machines to test compatibility with Power8* and s390x* architectures. We currently have the s390x systems verifying changes on a few of the main Kata repositories. The Power8 slaves are available and we are working to add jobs for this architecture to our CI.

Microsoft Azure:

Azure is where most of our CI jobs run; we run hundreds of VMs on a monthly basis for our testing here. Azure has been very stable, provides a lot of operating system images which we need for our testing, and the Jenkins-Azure VM plugin has worked perfectly for our use case. From jobs which test pull-requests to running Kubernetes* e2e conformance testing, we have heavily utilized Azure. In conjunction with GitHub* Actions, we have also explored utilizing Azure Kubernetes Service (AKS*) engine to create a containerd* based Kubernetes cluster to test kata-deploy image updates.

Packet:

When testing performance metrics, utilizing a bare metal instance is crucial to getting stable and usable results. To this end, Packet has helped provide bare metal infrastructure, via the CNCF Community Infrastructure Lab, on which we run our metrics CI tests. This allows us to keep track of key Kata metrics for each pull-request submitted. In addition, with Packet’s ARM* node support, we can make sure Kata continues to work well on ARM for each pull-request.

Vexxhost:

Vexxhost, an OpenStack-powered public cloud, has been very helpful since the beginning of the Kata Containers project. The base of our CI is driven by Vexxhost. Early on, Vexxhost began providing nested virtualization infrastructure support in their cloud, as they were very interested in being able to support Kata Containers. Vexxhost has provided and received a lot of feedback and support, resulting in Kata running smoothly on their cloud.

openSUSE* Open Build Service* (OBS):

The OBS platform has given us the flexibility to create packages for multiple Linux distributions, allowing us to distribute our packages minimizing the effort needed compared to building packages on our own. It helps us to build Kata packages in 30 to 60 minutes for several distributions. Currently we provide packages for Centos*, Debian*, Fedora*, openSUSE*, RHEL*, SLES* and Ubuntu*.

PackageCloud:

In addition to CI, we are also beginning to leverage PackageCloud to facilitate efficient management of distribution packages created by Kata. Their service donation allows the Kata community to focus on an optimal container runtime rather than efficient, user friendly artifact distribution. In addition to being a great service provider, they also have an awesome technical blog — checkout the illustrated guide to linux networking stack!

Get involved:

Explore Kata Containers on GitHub, KataContainers.io and get involved! Connect with the community via Slack or IRC Freenode: #kata-dev, Mailing List, Weekly meetings and Twitter.

*Other names and brands may be claimed as the property of others.