I’m very tempted to buy this https://www.etsy.com/listing/86833090/11x17-poster-robot-print-poster-monster

Programmable Infrastructure in Production with Kubernetes

Anne Currie
Mar 13, 2017 · 4 min read

This week I’m going to give a talk on automation and programmable infrastructure and I need a simple real-world example. I’ll use our Microbadger service, which we partially automated with our own custom scheduler. The code for our scheduler is open source and available at https://github.com/microscaling/microscaling. It scales by calling the K8s deployments API.

What does our service do?

Microbadger is a SaaS that lets you navigate the metadata for any public image on Docker Hub. You give us the image name, we get the data from Docker Hub, tidy it up and show it to you. So far, so standard.

How does it work?

Imagine a user requests a new image that isn’t already cached. There’s lots of data we want to show. Docker Hub gives us some of it very quickly and some takes much longer (in particular, a deep inspection that gets the download size and reverse engineers the docker command for each layer).

We decided to display the quick stuff ASAP to keep the user happy and just fill in the slow stuff when we got it, so we split our data requests and processing.

Everything inside the orange box below is either

So, what have we got?

Each Go service is single threaded, we just scale them up by running more pods, which for us just contain a single container. The Size and Inspector services share the same physical infrastructure (they run on the same AWS instances/nodes).

To start with, we just ran 10 Inspector pods and 10 Size pods split over 2 T2 Medium AWS Instances. Easy peasy, but a bit unexciting and massively overprovisioned most of the time to handle peaks. Could we use programmable infrastructure to be smarter and save ourselves some money?

Behaviourally, what are the key features of this system?

Ideal for programmable infrastructure!

This fairly standard architecture is perfect for optimising programmatically. To do this we used our Microscaling custom scheduler for k8s, which takes 3 inputs

The custom scheduler just creates inspector containers until the primary demand metric targets are met (the inspection queue isn’t backed up). It frees up the space for the new inspectors by killing off size service containers (remember they share the same machines). When the inspector queue length falls, the custom scheduler kills off inspectors and starts enough size services to keep the size queue managed.

This programmable infrastructure approach has several good features

Philosophy

Money saving is good, but what interests us is the change in philosophy. In a monolithic world all of these components would have been inside a single huge service, which would probably have done all of this custom scheduling itself internally (monitor internal queues and start/stop threads).

We split our service apart for the usual small-scale Microservice reasons

The downside of Microservices, however, is you lose the omniscient internal view and control that you can build into a monolith. This usually means over-provisioning is required. However, with custom schedulers & orchestrators we got back this oversight.

Microservices can be an architecture for the resource-rich and time poor. We used our custom scheduler to get some of the resource back without having to re-architect our services, which was nice!

Thanks for reading.

Anne Currie & Ross Fairbanks

Thanks to Ross Fairbanks.

Anne Currie

Written by

SciFi author interested in tech, engineering, science, art, SF, economics, psychology, startups. Chaotic evil.