Introducing greedy container instances!

Cirrus CI has a variety of supported compute services where it can schedule and orchestrate tasks on. But the most popular instances Cirrus CI orchestrates are the one running in a Kubernetes cluster: either it’s the managed by us “container instances” or a cluster that a user connected (either Google’s GKE, AWS’s EKS or Oracle’s OKE).

When a task that uses one of the above integrations is triggered, Cirrus CI uses Kubernetes API to create a pod and relies on the corresponding managed cluster for finding out resources and starting the container (including scaling the cluster up or down depending on the load). Such Kubernetes clusters are always a little over provisioned and do not fully utilize all available resources but on the bright side it means that starting a new container is just a matter of seconds, there is always some resources available to pick new tasks right away.

Recently we’ve been thinking how to utilize such idle resources and now we have an answer! Greedy tasks!

First, let’s dig deeper into how pods are scheduled on a cluster. In this particular case we are interested in resources that a pod can specify. Essentially there are two types of limits:

1. requests — is the guaranteed amount of CPU/Memory a pod will have access to.
2. limits — the maximum amount of CPU/Memory a pod can get access to if a containing Kubernetes node has unclaimed resources.

When you configure let’s say a container with two CPUs:

task:
  container: # can be gke_container, eks_container or oke_container
    image: golang:latest
    cpu: 2
    memory: 8G

Cirrus CI will create a pod with both requests and limits equal to the same value of 2 CPUs and 8Gb of memory. Which will guarantee that your task will always get 2 CPUs, no more no less.

For greedy tasks Cirrus CI will not set the CPU limit. Meaning that your task will be able to overcommit and use more CPU resources as requested if there are CPU available on a node where task is executing. Once a new task is scheduled on the node CPU, your over committed task will be throttled to claim back resources.

To make a container instance greedy just add greedy: true field to your *_container instance like so:

task:
  name: Test Greedy 🎉
  timeout_in: 3m
  container:
    image: containerstack/cpustress:latest
    cpu: 1
    memory: 1G
    greedy: true
  stress_script: stress-ng --cpu 4 --timeout 180

The example above requests at least 1 CPU but will try to use 4 CPUs. From the CPU chart of a task run for the above configuration we can see that it was able to use 2.5 CPU right from the beginning and then even 3.5 CPUs for some time!

CPU chart for a greedy task that used more CPUs than requested

Greedy instances are free for container instances managed by us:

• Greedy instance doesn’t require more compute credits. The amount of credits used by a task will be based on the CPU amount requested and not used.
• Greedy instance doesn’t contribute to the CPU limits one can use for the Open Source projects. The limits are based on the amount of CPUs your tasks are requesting.

If your tasks can benefit from occasionally more CPUs please give greedy instances a try! And please let us know how it went either on Twitter or GitHub.