Bin packing with two bins

Kubernetes at the opposite of scale

I’ve recently been daydreaming of what life would be like with $2 billion worth of cloud computing resource; fantasies include blowing it all on 23 million static IP addresses. Today we all discovered that big companies have similar daydreams, and are prepared to act on them.

I don’t know what Snap uses to manage their infrastructure, but here at Quartic we’re in love with Kubernetes. After just a couple of weeks of investment and a metric ream of YAML, our CEO can now rebuild and rehydrate our entire platform stack. In two minutes, with a single command.

That said, some of the Kubernetes utopia seems a little foreign. For example, here’s the pitch for “automatic binpacking”:

Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.

Resource-saving sounds like something we want. If we had 300 service replicas each needing on average 1 vCPU, Kubernetes would be stellar at distributing them across a 100-node 4-vCPU cluster. With capacity for rolling updates.

So then, why do we see this kind of thing all the time?

>> kubectl get po --all-namespaces
NAMESPACE     NAME                            READY     STATUS
abacus        catalogue-1208971147-db4nn      1/1       Running
abacus        howl-2411159971-1dvpt           1/1       Running
abacus        management-2370871570-z65qv     0/1       Pending  <<<
abacus        weyl-3998498193-2t81b           1/1       Running
core          ingress-controller-h28bf        1/1       Running
core          kube-lego-2314758353-86s13      1/1       Running
core          prometheus-2790691695-b0rsb     3/3       Running
core          taijitu-885483514-qdjgv         1/1       Running
diggers       tracker-3448647637-rvg1d        1/1       Running
[...]

>> kubectl describe po -n abacus management-2370871570-z65qv
[...]
Events:
  Type     Reason             Message
  --------  ------            -------
  Warning  FailedScheduling   pod failed to fit in any node
fit failure summary on nodes : Insufficient memory (2)

(For those unaccustomed to Kubernetes output, this says that one of our services can’t be scheduled on the cluster due to memory pressure.)

The answer is that this kind of bin-packing relies on the law of large numbers. At Quartic, we only have a small number. We’re still at the stage where we’re closer to 2–3 nodes per cluster. Despite building a SaaS, a 100-node cluster isn’t reasonable yet; our customers have data security concerns around multi-tenancy, and HA isn’t a requirement.

Dealing with small numbers

This problem will amortise away as we scale out. Right now though, how do we cope with this? Our palette so far consists of the following:

1. The sledgehammer approach. Provision a bigger cluster, eat the cost, and move on to other problems.
2. The whittling approach. Many sidecar services don’t need anything close to a full CPU, nor a ton of memory. Kubernetes exposes Docker’s resource-constraints system, and is there to be tuned.
3. The obsessive approach. Actual optimisation. Spend time looking at our worst offenders, and figure out how to make them better.

There’s a natural tension between these approaches, especially in early-startup mode. Do we eat into our runway via burn rate (#1) or via time (#3)?

In our case, we’ve largely managed to resist sledgehammering so far. Whittling gave us a substantial early win, though it’s likely a one-time boost. Obsessing over optimisations has provided us the biggest ongoing improvement, and has helped us substantially improve our internal architecture as a side effect. That said, this is also a known trap in early stages. I’ll be talking more about our optimisation efforts in a follow-up post.

In conclusion

A cluster manager like Kubernetes is well within the reach of small, lean teams with cloud-based infrastructure, and it quickly provides ongoing wins. However some of the benefits are only unlocked at scale, and you’ll likely need to make compromises until you get there. Every team has its own appetite for cost control vs. time control (our anecdote is that optimising on cost worked out), but you’ll have options either way.

We’re more than happy to chat in more detail to others in a similar position; feel free to reach out to me at oliver@quartic.io!

About me

I’m Oliver Charlesworth, head of engineering at Quartic Technologies. We’re building a platform to bring real-time business analytics to operations in the field.